Skip to main content

5.7 Building the Kernel and Modules with Clang

Now, let's step into the world of Clang.

Remember the cliffhanger at the end of the previous section? Certain out-of-bounds defects, such as "left-out-of-bounds" access on global variables, can only be caught when compiled with Clang 11 or later. This isn't magic—it's simply a difference in compiler capabilities.

Why Switch Compilers?​

Before we get our hands dirty, let's discuss why we're doing this.

The LLVM project (short for Low Level Virtual Machine) might have "Virtual Machine" in its name, but it has nothing to do with traditional virtual machines anymore. It's a powerful compiler backend infrastructure. Clang (rhymes with "slang") is a modern C/C++ compiler frontend built on top of LLVM.

You can think of Clang as a modern replacement for GCC.

But GCC is still going strong, so why switch? The reasons are practical:

  1. Better diagnostics: Clang's error messages are much clearer than GCC's, which can be a lifesaver when debugging.
  2. Catches certain bugs more accurately: Going back to our cliffhanger—for the "left-out-of-bounds" access on global memory we mentioned in the previous section, GCC (versions 9.3, 10, and 11) reliably missed it, but Clang caught it.

This point is crucial. It means that to thoroughly sweep out memory blind spots, we need this new weapon in our arsenal.

Don't Mix Compilers (A Hard-Learned Lesson)​

Before we continue, consider this a warning.

When my first KASAN test failed, I was completely stumped. I reached out to Marco Elver, one of the primary maintainers of KASAN. He pointed out a rookie mistake I had made: I was trying to compile a kernel module with Clang, but the target kernel itself was compiled with GCC.

This is an absolute no-go.

The kernel and its modules must be built by the same compiler. Why? Because the underlying Application Binary Interface (ABI) must be perfectly consistent. If the compilers differ, struct alignment, calling conventions, and even function name mangling might not match up. This mismatch leads to inexplicable crashes—or, as in my case, causes KASAN to completely fail.

So, the rule is simple: either all GCC or all Clang. Since we want to catch those tricky bugs, let's recompile both the kernel and the modules with Clang 11 (or later).

Preparation: Installing Clang​

If you're still on Ubuntu 20.04 LTS, you'll need to manually install Clang 11.

sudo apt install clang-11 --install-suggests

If, like me, you have both Clang 10 and Clang 11 sitting on your system, I recommend manually creating a symlink to point llvm-objdump to version 11 to avoid confusion:

sudo ln -s /usr/bin/llvm-objdump-11 /usr/bin/llvm-objdump

"The Better Path" — Just Use Ubuntu 21.10

Honestly, manually wrestling with versions on an older system is a quick way to lose your hair. There's a more hassle-free approach: just install Ubuntu 21.10 (or later) as a new x86_64 virtual machine.

Ubuntu 21.10 comes with Clang 13 out of the box. That's exactly what I did—spun up a new VM and saved myself the agony of dependency hell. For the following steps, whether you use Clang 11 or Clang 13, the process is exactly the same.

Building: Swapping GCC for Clang​

Alright, tools are ready. Time to make the cut.

We're going to compile that 5.10.60 kernel. This is very similar to what we did in Chapter 1, with the only difference being that we explicitly specify the compiler in the make command.

Switch to your kernel build directory and run:

$ time make -j8 CC=clang
  SYNC    include/config/auto.conf.cmd
*
* Restart config...
* Memory initialization
*

The First Difference You'll Encounter​

When you run this command with CC=clang for the first time, the kbuild system will detect the change in environment. Clang supports some advanced features that GCC doesn't have, so the system will pause and ask you about some new configurations.

First, it will ask if you want to automatically initialize kernel stack variables:

Initialize kernel stack variables at function entry
> 1. no automatic initialization (weakest) (INIT_STACK_NONE)
  2. 0xAA-init everything on the stack (strongest) (INIT_STACK_ALL_PATTERN) (NEW)
  3. zero-init everything on the stack (strongest and safest) (INIT_STACK_ALL_ZERO) (NEW)
choice[1-3?]:

This is a very tempting option—enabling it helps you nip a huge number of "Uninitialized Memory Read (UMR)" bugs in the bud.

But wait!

Our goal is to test whether KASAN and UBSAN can catch these bugs on their own. If the compiler initializes all memory for us, the bugs get masked. So, to verify our toolchain, I intentionally chose the default option 1 (do not auto-initialize).

Next, the build process will ask you a few questions about heap memory initialization, such as whether to zero out memory on allocation or free. I kept the defaults for these by just pressing Enter (or choosing y, since this won't mask bugs other than UMR):

Enable heap memory zeroing on allocation by default (INIT_ON_ALLOC_DEFAULT_ON) [Y/n/?] y
Enable heap memory zeroing on free by default (INIT_ON_FREE_DEFAULT_ON) [Y/n/?] y
*
* KASAN: runtime memory debugger
*
KASAN: runtime memory debugger (KASAN) [Y/n/?] y
  KASAN mode
  > 1. Generic mode (KASAN_GENERIC)
  choice[1]: 1
 [...]
  Back mappings in vmalloc space with real shadow memory (KASAN_VMALLOC) [Y/n/?] y
  KUnit-compatible tests of KASAN bug detection capabilities (KASAN_KUNIT_TEST) [M/n/?] m
  [...]

If all goes well, the kernel will finish compiling. The remaining steps should be familiar to you—install the modules, then install the kernel.

Don't forget that you also need to include CC=clang on the command line:

sudo make CC=clang modules_install && sudo make CC=clang install

Once that's done, reboot your virtual machine and select your newly compiled kernel from the GRUB menu.

Verification: Was It Really Compiled with Clang?​

After booting into the system, the first thing to do is confirm you aren't dreaming. Run this:

$ cat /proc/version
Linux version 5.10.60-dbg02 (letsdebug@letsdebug-VirtualBox) (Ubuntu clang version 13.0.0-2, GNU ld (GNU Binutils for Ubuntu) 2.37) #4 SMP PREEMPT Wed ...

Look at the line in parentheses—Ubuntu clang version 13.0.0-2.

Seeing this confirms that the kernel was indeed translated by Clang. To tell them apart, I added -dbg02 to the uname of the Clang-compiled kernel, while I called the GCC-compiled version 5.10.60-dbg02-gcc. This way, I won't get mixed up later when ls /lib/modules.

The Final Piece of the Puzzle: Compiling the Module​

The kernel is ready. Now it's time for our test module, test_kmembugs.

Switch to the source directory and pass the CC variable to make:

cd <book_src>/ch5/kmembugs_test
make CC=clang

Simple, right?

To save myself some trouble, I've actually already written this logic into the load_testmod script—it automatically detects which compiler the current kernel was built with, and then uses that corresponding compiler to build the module. This way, I don't have to manually specify it every time.

Time to Practice

As the saying goes, reading about it is never as good as doing it.

Now it's your turn:

Exercise: Build a debug kernel from scratch using Clang, and use it to compile our test_kmembugs.ko module. Run through the previous test cases, especially that "left-out-of-bounds" bug that GCC missed, and see how Clang roots it out.

This concludes the first part of our deep dive into kernel memory defect detection.

With KASAN, UBSAN, and the sharp sword of Clang in our hands, our arsenal is now quite complete. Next, we'll do a retrospective—putting all the tools and techniques we've used together, carefully comparing their pros and cons, and looking at which approach to use in which scenario.

That brings us to the next section: "Catching memory defects in the kernel – comparisons and notes (Part 1)". A big table is waiting for you there.