Affinity Tailor: Dynamic Locality-Aware Scheduling at Scale
Google's new CPU scheduler squeezes 12% more throughput from modern chiplet processors by keeping related work physically close together in silicon.
The Thesis
Modern servers increasingly use 'chiplet' designs — chips built from multiple smaller silicon dies stitched together — where crossing from one die to another costs extra memory access time. Linux's default scheduler, called CFS (Completely Fair Scheduler), ignores this topology and happily moves work across die boundaries to keep all CPU cores busy. Affinity Tailor, deployed inside Google's production infrastructure, fixes this by estimating how many CPU cores each workload actually needs at any moment and then assigning it a compact, topologically local slice of cores as a soft preference rather than a hard wall. The result is 12% more work done per CPU on chiplet systems, with the gains coming from better cache reuse and less cross-die memory traffic. The catch: this only works at the scale and observability Google already has, and reproducing it elsewhere requires real-time demand estimation infrastructure that most organizations lack.
Catalyst
Chiplet-based server CPUs — AMD EPYC and Intel's forthcoming disaggregated designs — became mainstream in data centers only in the last two to three years, making cross-die locality a newly urgent problem at scale. At the same time, cloud providers now run densely packed, multi-tenant workloads where the cost of wasted CPU cycles compounds across millions of cores. Google's internal telemetry and fleet-management infrastructure (built over the past decade) finally provides the per-workload demand signal that makes online demand estimation tractable.
What's New
Prior approaches fell into two camps: Linux CFS, which load-balances aggressively and ignores topology, or hard CPU partitioning (cpusets), which reserves cores statically and leaves them idle when workloads underutilize their allocation. Earlier research systems like Heracles and Parties explored resource partitioning for latency-sensitive workloads but required manual tuning or coarse-grained controls. Affinity Tailor's key departure is treating topology-aware CPU sets as soft affinity hints rather than hard partitions — the kernel still allows work to spill outside the preferred set when necessary, so utilization is preserved while locality is improved.
The Counter
The 12% throughput gain is real at Google, but Google is not a typical organization. Their demand-estimation system relies on years of fleet telemetry, workload classifiers, and internal APIs that no public cloud customer or on-premises operator can replicate from a paper. The results are measured on Google's specific workload mix — search, ads, and internal batch jobs — which may have unusually strong locality properties compared to, say, a mixed SaaS workload or a scientific computing cluster. The paper also acknowledges that gains on non-chiplet systems drop to 3%, which is modest enough that many operators would not justify the engineering cost of deploying a custom userspace scheduler controller. Furthermore, chiplet vendors are themselves improving cross-die interconnect bandwidth and coherency protocols with each generation, which could erode the locality penalty that Affinity Tailor exploits. The upstream Linux kernel community has historically been slow to adopt userspace-guided scheduling hints, so the path to broad adoption outside Google's proprietary stack is unclear.
Longs
- AMD (AMD) — EPYC chiplet CPUs are the primary hardware beneficiary; better scheduling makes AMD's multi-die architecture more attractive
- BOTZ (Global X Robotics & AI ETF) — indirect exposure to data center efficiency gains
- Ampere Computing (private, ARM-based cloud CPUs) — locality-aware scheduling advantages apply to their NUMA designs
- Pure Storage (PSTX) — faster CPU execution reduces memory residency, compressing storage I/O patterns in ways that favor NVMe-optimized vendors
- Cloudflare (NET) — edge compute operators with dense multi-tenant workloads stand to benefit from scheduler efficiency gains
Shorts
- Linux CFS maintainers / Red Hat — their default scheduler is shown to leave 12% performance on the table on chiplet systems, creating pressure to overhaul upstream Linux scheduling
- VM-based cloud isolation vendors — if soft affinity hints prove as effective as hard partitions, the case for expensive VM-level isolation weakens
- Intel (INTC) — Intel's UPI interconnect architecture is less chiplet-fragmented than AMD's, so Intel's relative advantage in cross-die latency narrows if software can compensate for AMD's topology
Enablers (Picks & Shovels)
- Linux kernel's cgroup and cpuset interfaces — the underlying kernel primitives Affinity Tailor builds on
- AMD EPYC and Intel Xeon processors with NUMA/chiplet topology — the hardware where gains are largest
- Google's Borg/fleet management telemetry — the demand-estimation infrastructure that makes online workload sizing possible
- perf and hardware performance counters — necessary for measuring LLC (last-level cache) miss rates that guide scheduling decisions
Private Watchlist
- Ampere Computing — ARM-based cloud CPU designer whose multi-core topology faces the same scheduling challenges
- Nscale — European GPU/CPU cloud startup optimizing dense multi-tenant compute
- Oxide Computer — hyperscaler-adjacent hardware company building rack-scale systems where topology-aware scheduling is a first-class concern
Resources
The Paper
Modern large multicore systems often run multiple workloads that share CPUs under schedulers such as Linux CFS. To keep CPUs busy, these schedulers load-balance runnable work, causing each workload to execute on many cores. This weakens locality at the microarchitectural level: workloads lose reuse in caches, branch predictors, and prefetchers, and interfere more with one another - especially on chiplet-based systems, where spreading execution across cores also spreads it across LLC boundaries. A natural alternative is strict CPU partitioning, but hard partitions leave capacity idle when workloads do not fully use their reserved CPUs. We present Affinity Tailor, a userspace-guided kernel scheduling system built on a key insight: the kernel can preserve locality for workloads that share CPUs by treating demand-sized, topologically compact CPU sets as affinity hints rather than hard partitions. A userspace controller estimates each workload's CPU demand online and assigns a preferred CPU set sized to that demand, chosen to be as disjoint as possible from other workloads while spanning as few LLC domains as possible. The kernel then uses this set as an affinity hint, steering threads toward those CPUs while still allowing execution elsewhere when needed to preserve utilization. Deployed at Google, Affinity Tailor delivers geometric-mean per-CPU throughput gains of 12% on chiplet-based systems and 3% on non-chiplet systems over Linux CFS. Furthermore, faster execution reduces memory residency, yielding per-GB throughput gains of 3-7%. Our findings suggest that future schedulers should treat spatial locality as a first-class objective, even at the expense of work-conservation.