Uber Moves beyond CPU Cores with Performance-Aware Scheduling for Stateful Workloads

Uber has introduced a scheduling scheme called Normalized Compute Units (NCUs) in Odin, its platform for orchestrating stateful workloads, to account for the fact that CPU cores on different servers do not deliver equal performance. Craig Risi reported the details for InfoQ. Odin spans roughly 5 million CPU cores across on-premises data centres and public clouds, covering several generations of x86 hardware as well as ARM processors, and Uber measured single-threaded performance differing by nearly a factor of two between its fastest and slowest cores.
That spread creates a resource-allocation dilemma. Sizing workloads for the weakest hardware leaves capacity idle on faster machines; sizing to an average leaves workloads short of CPU when they land on slower servers. For stateless services, load balancing can shift traffic toward faster or less-loaded machines and soften the problem. Stateful systems have no such escape hatch, because a request often must reach the server holding a particular shard. Uber describes workloads where a leader uses more than ten times the CPU of its followers, so relocating that leader from a fast server to a slow one can sharply raise its consumption without any change to the workload, risking resource pressure.
The NCU model separates how much compute performance a workload needs from how many physical cores supply it. Odin assigns every server type a performance factor, with an older hardware generation as the baseline of 100. Newer machines reach considerably higher values, with some approaching 190. Factors are first set during hardware qualification and then adjusted using measurements from production workloads, rather than resting on synthetic benchmarks alone. Uber says its Kafka measurements showed markedly different performance across ARM, older on-premises hardware and newer x86 systems, which is why it keeps refining the numbers from live data.
With factors in place, observed CPU usage converts into normalized units. Odin takes a workload's requirement in compute capacity and translates it into a core count that varies by host: fewer cores on fast hardware, more on slow hardware. The goal is comparable capacity, not identical core counts. The design choice to re-measure in production is significant for anyone copying the idea, because vendor benchmark suites rarely capture the interaction between a specific workload and a specific microarchitecture, such as cache behaviour or memory bandwidth effects.
Uber has applied performance-aware allocation to about 1.4 million CPU cores of automatically scaled Odin workloads. For Kafka, CPU utilisation initially varied by roughly eight percentage points between server types, with the hottest hardware at around 50% higher utilisation than the coldest. After the change, the gap fell to about three percentage points, with the hottest hardware only around 10% above the slowest. Uber also cut total Kafka CPU allocation by approximately 8%, with much of that reduction falling on faster hardware, where the freed capacity is arguably the most useful.
This suggests the appeal for Uber is twofold: smoother utilisation reduces the risk that a stateful leader on slow silicon starves, and reclaiming allocation on fast silicon frees capacity for other work. An 8% reduction in allocated cores across a large Kafka estate is a meaningful amount of infrastructure, though the published figures cover utilisation spread and allocation rather than latency or reliability outcomes.
The approach differs from the CPU abstraction Kubernetes uses. Kubernetes expresses CPU requests and limits in cores or millicores, deliberately treating a CPU unit as a fixed quantity of CPU time irrespective of the machine underneath. That keeps workload definitions portable across clusters, but it does not encode the relative speed of different processor generations. Uber's NCU model asks a different question: how much computational capability a given allocation actually represents on this host.
In practice, teams running heterogeneous fleets face a similar trade-off. Performance-weighted units make capacity planning more honest, but they add a maintenance burden: every hardware type needs a trustworthy performance factor, and those factors drift when firmware, kernel versions or workload mixes change. Factors also have to be scoped carefully. A factor tuned for Kafka's access patterns may not suit another stateful service, which is presumably why Uber updates its numbers from production data per technology rather than maintaining one global table.
A likely trade-off is predictability. With core-based requests, a developer knows exactly what the scheduler will reserve; with normalized units, the same request maps to different core counts on different hosts, which complicates capacity forecasting, chargeback and debugging. Uber's NCU work appears to sidestep that by keeping the conversion inside Odin and applying it to automatically scaled workloads, so the abstraction stays in the orchestrator rather than in every service's resource declaration.
Why it matters: Operators of large mixed-generation fleets, especially those running stateful services such as Kafka, are the most directly affected, because they cannot rebalance their way out of performance variance the way stateless services can. For developers and platform teams, the result points toward treating CPU requests as a performance contract rather than a core count, at least where hardware heterogeneity is unavoidable. Whether the pattern generalises beyond Uber's Odin setup will depend on how cheaply performance factors can be measured and kept current, which is an open question rather than something the reported results settle.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Uber has introduced Normalized Compute Units (NCUs) to make CPU allocation more consistent across its highly heterogeneous server fleet. By Craig Risi