CppCon2025

Cache-Friendly C++

Jonathan Müller

YouTube

These are notes from Jonathan Müller's CppCon 2025 talk. Jonathan has long worked on low-latency C++ (at think-Cell at the time of the talk, later at LSEG on HFT market-data feeds). He starts from a counter-intuitive phenomenon that raises everyone's blood pressure: std::unordered_set lookup is O(1), std::vector linear lookup is O(n), yet on a real machine, at small sizes vector beats unordered_set. There is only one direction for the answer: the CPU cache.

This talk is the sister of the microbenchmark one. Part three of that series already pointed at the branch predictor and cache conspiring, but did not expand on the cache itself. This talk takes the cache apart from the ground up: why it exists, the unit it moves data in, how much each level costs, and how every decision you make in C++ — picking a container, a data type, a struct layout — lands on cache behavior.

The notes are split into four parts, moving from "breaking the complexity religion" through "building cache intuition" to "landing it in code." All experiments were run on the same machine: Arch Linux / WSL2, AMD Ryzen 7 9700X (Zen 5), GCC 16.1.1, -std=c++20, with cache hierarchy L1d 48 KiB/core, L2 1 MiB/core, L3 32 MiB shared. Your numbers will differ; the direction of the conclusions will not.

Contents

Complexity Is Not Everything: How O(1) Lost to O(n)→Memory Is 100x Slower — For Real: The Cache Hierarchy and Cache Lines→Data Types Are a Cache Variable Too: Shrinking the Type Isn't Always Faster→Writing Cache-Friendly Code: Layout, Alignment, and Decisions→