Cloudflare's Gen 13: Powering the Future Internet

Cloudflare's Next-Gen Server Architecture

Cloudflare's detailed breakdown of their Gen 13 server, 'Gen 13: how we built our most powerful server yet,' offers a compelling look into their infrastructure scaling strategy, particularly in light of their FL2 rewrite in Rust. The decision to adopt the AMD EPYC™ 9965 Turin processor, despite its reduced L3 cache per core, is a well-justified pivot driven by the evolving nature of their workloads, which now scale more favorably with core count. This highlights a crucial trend in high-performance computing where workload characteristics dictate optimal hardware choices, moving beyond traditional metrics like cache. The doubling of memory capacity and network bandwidth, coupled with the adoption of PCIe 5.0 storage, demonstrates a holistic approach to performance enhancement, aiming for significant gains in throughput and efficiency. The emphasis on operational simplicity through higher-density servers is also a key takeaway, reflecting the challenges of managing massive global infrastructure.

However, while the article meticulously details the 'why' behind each component, a deeper dive into the specific performance regressions or challenges encountered during the evaluation of alternative AMD Turin SKUs (9755, 9845) would have provided even richer context. Understanding the precise trade-offs that made the 9965 the clear winner, beyond the general statement of 'performance per total cost of ownership (TCO)', could offer valuable lessons for other organizations making similar hardware decisions. Furthermore, while the move to 100 GbE is justified by future-proofing and industry economics, the article could benefit from more granular data on the projected traffic growth and how the 200 Gb/s aggregate bandwidth specifically addresses those future demands. The discussion on storage, particularly the NVMe 2.0 features like ZNS, is insightful, but more concrete examples of how these features translate into tangible benefits for Cloudflare's specific services (Durable Objects, Containers, Quicksilver) would enhance its impact.

Key Points

• Cloudflare has launched Gen 13, its most powerful server to date, designed to support the FL2 (Rust-based core request handling layer) rewrite.
• Key hardware upgrades include a 192-core AMD EPYC™ 9965 Turin processor, 768 GB DDR5-6400 memory, and 24 TB of PCIe 5.0 NVMe storage.
• The choice of the 9965 processor is driven by FL2's reduced L3 cache dependency and increased scalability with core count, offering up to 2x throughput and 50% better performance/watt.
• Memory capacity has doubled to 768 GB, utilizing all 12 DDR5-6400 memory channels for increased bandwidth, and maintaining a 4 GB/core ratio for future growth.
• Storage capacity increased to 24 TB with PCIe 5.0 NVMe drives, featuring an added front drive bay for up to 10 U.2 drives and support for NVMe 2.0 specifications.
• Network bandwidth has quadrupled with dual 100 GbE OCP 3.0 NICs, addressing the bottleneck identified with the previous dual 25 GbE configuration.
• The design prioritizes operational simplicity through higher-density servers and maintains a dual-vendor strategy for NICs to ensure supply chain resilience.

---

📖 Source: Inside Gen 13: how we built our most powerful server yet