Skip to main content

1.7 Building Your Custom Production Kernel

At this point, I'll assume you're no longer a stranger to the basic workflow of building a Linux kernel from source: fetching the source tree, configuring, and compiling. If you feel a bit rusty or want to brush up on the details, I highly recommend flipping through Linux Kernel Programming or checking out the "Further Reading" section at the end of this chapter.

Since we're talking about a "production kernel," our goal is clear: it must be stable, fast, and secure. But as we discussed in the previous section, we shouldn't just guess configurations from scratch. The smartest approach is to "stand on the shoulders of giants"—tuning based on an existing system's configuration (the so-called localmodconfig strategy). Once we have this reliable starting point, we further enhance security through "hardening" configurations.

Let's take the first step and lay a solid foundation.

1.7.1 Fetching the Source and Base Configuration​

Goal: Set up a working directory, fetch and extract the LTS kernel source.

Why do this: In the previous section, we selected LTS version 5.10.60 as our baseline. To keep our environment clean, we need a dedicated working directory to store the source code and build artifacts. The --directory parameter of tar lets us place files in the right location while extracting, saving us a few extra mv commands.

Working Directory: User home directory

Commands and Output:

First, let's set up a dedicated workshop:

mkdir –p ~/lkd_kernels/productionk
cd ~/lkd_kernels

Next, let's fetch the source. Here we use wget to grab the compressed archive directly, though you could also use git (if you don't mind a slightly longer wait):

wget https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/linux-5.10.60.tar.xz

Extract it:

tar xf linux-5.10.60.tar.xz --directory=productionk/

Enter the extracted directory and verify the version information along the way. This might seem like a boring step, but there's actually something quite interesting here:

Every version of the Linux kernel has a codename—the codename for 5.10.60 is "Dare mighty things". I quite like this name; it's very fitting for what we're doing here, as we are essentially performing surgery on the very heart of the operating system.

1.7.2 Generating a Trimmed Configuration (localmodconfig)​

Goal: Use the list of modules currently running on the system to generate a tailored initial kernel configuration file.

Why do this: A full default configuration includes thousands of drivers and modules you don't need, resulting in a bloated and slow kernel. The magic of localmodconfig is that it scans which modules are currently loaded on your system (the output of lsmod) and turns off everything that isn't selected. It's like performing liposuction on the kernel.

Working Directory: Kernel source root directory (~/lkd_kernels/productionk/linux-5.10.60/)

Commands and Output:

First, generate a snapshot of the current system's modules:

lsmod > /tmp/lsmod.now

Then feed this snapshot to make:

make LSMOD=/tmp/lsmod.now localmodconfig

⚠️ Pitfall Warning During this process, the terminal might pause and ask you a few questions (interactive interface). Don't panic—if you don't understand the specific meaning of an option, just press Enter to accept the default value, which is usually the safest bet. Our goal here is simply to get a "good enough" starting point, not to finalize every detail right now.

Once this step is done, you'll find a new .config file in the root of the source tree. This is a treasure—it's the "soul" of the kernel.

Back it up first, just in case we break something later and need to revert:

cp –af .config ~/lkd_kernels/kconfig_prod01

Tip: You can always type make help to see all available configuration commands. If you're a veteran, you can completely use make menuconfig to change the interface to a style you're familiar with.

1.7.3 Kernel Hardening—The Price of Security​

Goal: Use automated tools to check and modify the kernel configuration, enabling security hardening options.

Now we have a "working" configuration. But before putting it into production, we need to address a dilemma: security vs. convenience.

Many hardening features are disabled by default in the Linux kernel. Why? Because they either sacrifice performance or break compatibility with certain legacy software. But in production environments, especially for systems dedicated to debugging or running sensitive services, we lean towards "security."

The problem is: even if you open make menuconfig, faced with thousands of options, you won't know which ones control security.

Fortunately, someone has done this dirty work for us.

Introducing the Tool: kconfig-hardened-check

This is actually a validator—but it works in reverse to tell you what industry-recognized security best practices are missing from your configuration. It's like a strict security auditor going down a checklist against your .config and ticking off items one by one.

Install and run it (using the Python script as an example here):

# 假设你已经通过 pip 安装了该脚本，或者克隆了仓库
python3 kconfig-hardened-check.py --config .config

You'll see a ton of output, like this (excerpt):

[+] Check: CONFIG_BUG is required: set to y
[+] Check: CONFIG_STRICT_KERNEL_RWX is required: set to y
[-] Check: CONFIG_DEBUG_LIST is suggested: NOT set (should be y)
[...]

We don't need to dive deep into every detail of this script here (that deserves its own article). The core logic is: it tells you what you're missing, and you add it.

Based on its recommendations, I adjusted the final configuration for the production kernel. I've placed this configuration file in this book's GitHub repository (ch1/kconfig_prod01); you can use it directly or as a reference.

1.7.4 Compiling and Installing​

Goal: Compile the kernel, install the modules, and update the bootloader.

Now, the configuration is finally set in stone. It's time to turn this source code into that famous bzImage.

Working Directory: Kernel source root directory

First, confirm how many CPU cores you have—this determines how fast we can get this done:

$ nproc
4

I have 4 cores here. As a rule of thumb, the number following make -j is usually twice the number of cores. This maximizes CPU utilization while avoiding overload.

$ make -j8
[ ... 一大段滚动的编译日志 ... ]
BUILD   arch/x86/boot/bzImage
Kernel: arch/x86/boot/bzImage is ready  (#1)

When you see the line is ready, you've succeeded.

But there are two files here, and we need to clarify their relationship slightly:

$ ls -lh arch/x86/boot/bzImage vmlinux
-rw-r--r-- 1 letsdebug letsdebug 9.1M Aug 19 17:21 arch/x86/boot/bzImage
-rwxr-xr-x 1 letsdebug letsdebug  65M Aug 19 17:21 vmlinux
  • bzImage (9.1MB): This is the compressed kernel image. This is the file that GRUB actually reads and boots. Don't be fooled by the b in the name; it doesn't stand for Big, but rather "boot zImage" (roughly).
  • vmlinux (65MB): This is the uncompressed ELF executable. Don't delete it. While it's not used for booting, it contains all the symbol information—that's our lifeline when debugging later.

SICP-style aside: There's a subtle division of labor here. bzImage is for survival—it needs to be small enough to be loaded into memory and quickly decompressed; vmlinux is for understanding—it retains all the details for us to perform an autopsy when the system crashes.

Next, install the kernel modules. This copies the compiled .ko files into the system's standard directory, /lib/modules/$(uname -r):

$ sudo make modules_install
[ ... ]
  DEPMOD  5.10.60-prod01
$ ls /lib/modules/
5.10.60-prod01/  5.11.0-27-generic/  5.8.0-43-generic/

Notice that 5.10.60-prod01 directory—that's our new turf.

The final step is to let the system "know" this new kernel exists. We need to generate the initramfs (initial RAM filesystem) and update the GRUB configuration:

sudo make install

Several scripts actually run behind the scenes with this command. initramfs is absolutely critical; it contains a minimal set of drivers and scripts responsible for preparing the hardware environment before the kernel actually mounts the root filesystem on the disk. Without it, modern systems usually won't boot.

1.7.5 First Ignition—Switching to the New Kernel​

Goal: Reboot the system, select the new kernel from the GRUB menu, and verify the system is running normally.

Everything is ready; all that's left is to reboot.

But before hitting the restart button, pause for a second. Think about it: if this step succeeds, what do you expect to see?

You should see the GRUB boot menu. If your menu is normally hidden (or skips too quickly), you can call it up by mashing the Shift key (for legacy BIOS) or the Esc key (for UEFI) during boot.

You should see a new option named "5.10.60-prod01".

Select it and press Enter.

If you're working in a virtual machine, like my Oracle VirtualBox here, you might also notice an issue: the resolution is wrong, or the mouse isn't automatically captured. Don't worry—that's just because VirtualBox's Guest Additions haven't been compiled for the new kernel yet. This doesn't affect kernel functionality at all; it just makes the experience slightly less polished. To avoid distractions, I usually just SSH in to get work done at this point.

If all goes well, you'll see the familiar login prompt.

Once in the system, type this command to confirm we're sitting on the new engine:

$ uname -a
Linux dbg-LKD 5.10.60-prod01 #1 SMP PREEMPT Thu Aug 19 17:10:00 IST 2021 x86_64 x86_64 x86_64 GNU/Linux

That 5.10.60-prod01 suffix is the mark we made with our own hands.

Milestone Summary We now have a system running on our custom kernel. It's trimmed, hardened, and ready for action. But this is just a "production kernel"—it's like an agile but tight-lipped secret agent.

In the journey ahead, you'll encounter all sorts of tricky kernel-level bugs. Sometimes, you'll need to make this agent talk, or have it log every single thought process.

When that time comes, relying solely on this "production kernel" won't be enough.

We'll need another character—more chatty, perhaps even a bit bloated: the debug kernel.