DoorDash's 1.5M RPS Cache: Envoy & Valkey Powering 99.99999% Availability

Infrastructure-Level Caching Unpacked

DoorDash's Entity Cache represents a sophisticated evolution in microservice architecture, moving caching from an application concern to an infrastructure one via Envoy. The reported 99.99999% availability and 1.5 million RPS are staggering figures, highlighting the platform's robustness. The use of Kafka for event-driven invalidation, dual TTL thresholds for resilience during outages, and optimizations like custom buffer pools and a lock-free single-flight mechanism are particularly noteworthy. This approach significantly reduces complexity for individual service teams by abstracting caching logic, allowing them to focus on business functionality. The proactive refresh strategy using the XFetch algorithm is a clever way to combat cache stampedes and maintain high hit rates. The transparency of the proxy means minimal to no changes are required in existing service code, which is a massive win for adoption velocity.

However, while the article touts impressive performance metrics, it's crucial to consider the operational overhead. Managing a distributed system of this scale, with Envoy as a central component and Valkey as the distributed cache, requires significant expertise in observability, monitoring, and incident response. The complexity introduced by the infrastructure-level caching, though beneficial for developers, might present new challenges for SRE and operations teams. Furthermore, while the Kafka-based invalidation is elegant, it introduces a dependency on Kafka's reliability and latency characteristics. The 'dual TTL' approach, while enabling availability during outages, means serving potentially stale data – a trade-off that needs careful consideration depending on the criticality of the data being cached. The article could benefit from more detail on how cache consistency is managed across different regions or availability zones, and the strategies employed to mitigate potential data staleness issues beyond the dual TTL mechanism, especially in scenarios where strict consistency is paramount.

Key Points

• DoorDash has implemented Entity Cache, a transparent proxy caching platform using Envoy and Valkey.
• The platform handles over 1.5 million RPS with 99.99999% availability, reducing redundant service-to-service requests.
• Caching is moved to the infrastructure layer, requiring no application code changes for services.
• Key features include event-driven invalidation via Kafka, dual TTL thresholds for resilience, and optimizations like custom buffer pools and the XFetch algorithm.
• Optimizations led to significant improvements in allocation rates, per-pod throughput, and P99 latency spikes.

---

📖 Source: DoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability