Uber's NCUs: Smarter CPU Allocation for Stateful Apps

Beyond Raw Cores: Uber's NCU Innovation

Uber's introduction of Normalized Compute Units (NCUs) represents a sophisticated and much-needed evolution in resource scheduling for complex, large-scale environments. The core insight is profoundly simple yet impactful: CPU cores are not created equal. By accounting for the performance variations across diverse hardware generations and architectures (x86, ARM), Uber moves beyond the simplistic abstraction of raw core count. This is particularly critical for stateful workloads, where the inability to easily migrate active connections or data shards makes performance predictability paramount. The Odin platform's approach of assigning performance factors and translating workload requirements into a dynamic number of physical cores based on server capability directly tackles the waste and under-provisioning issues inherent in homogeneous CPU allocation. The reported improvements in CPU utilization consistency and overall allocation reduction for Kafka are compelling evidence of the system's effectiveness. This is a substantial technical advancement that addresses a pervasive problem in cloud and data center operations.

While the article highlights the benefits, a deeper dive into the ongoing calibration and maintenance of these performance factors would be valuable. How frequently are these factors updated? What is the process for detecting and incorporating new hardware or performance regressions? Furthermore, while the comparison with Kubernetes' millicore abstraction is apt, understanding the integration challenges or potential synergies between NCUs and existing orchestration layers like Kubernetes could offer further practical guidance. The success hinges on the accuracy and dynamism of the performance factors; any lag or inaccuracy could reintroduce the very problems NCUs aim to solve. Nevertheless, for organizations managing heterogeneous fleets and running performance-sensitive stateful services, Uber's NCU model offers a compelling blueprint for achieving greater efficiency and reliability.

Key Points

• Uber has developed Normalized Compute Units (NCUs) to address performance inconsistencies across its heterogeneous server fleet.
• NCUs decouple workload compute requirements from physical CPU cores, accounting for varying processor performance.
• This approach is crucial for stateful workloads where migrating traffic is difficult and performance predictability is key.
• NCUs help prevent resource waste on faster machines and under-provisioning on slower ones.
• Performance factors are determined through hardware qualification and refined with production workload data, including technology-specific measurements (e.g., Kafka on ARM vs. x86).
• Uber has seen significant improvements in CPU utilization consistency and achieved an ~8% reduction in total Kafka CPU allocation.

---

📖 Source: Uber Moves Beyond CPU Cores with Performance-Aware Scheduling for Stateful Workloads