Shopify's ClickHouse Observability: 100M Events/Sec

Global Scale Observability Unlocked

Shopify's successful migration to a self-hosted ClickHouse-based observability platform, Observe, is a compelling testament to the power of open-source databases for handling massive telemetry data at global scale. The reported 16x to 30x query performance improvement and substantial cost control over fragmented vendor solutions are significant achievements. The article highlights the crucial role of ClickHouse in unifying metrics, logs, traces, and exceptions onto a single engine, simplifying complexity and enabling faster incident detection, which is paramount given the potential $5.1 million cost of downtime per minute during peak periods. The technical details around managing flexible schemas with materialized views, optimizing ingestion pipelines with Kafka, and the continuous efforts in tiered storage and schema evolution demonstrate a mature engineering approach to a complex problem.

However, the article also subtly points to potential limitations and considerations. The initial decision to go with self-hosted ClickHouse, while providing control, clearly came with a significant operational burden. Elijah McPherson's retrospective suggestion to consider ClickHouse Cloud if starting today underscores the complexity of managing such a system at scale. While the article extols the virtues of open-source contributions and understanding the source code, the sheer scale of Shopify's operations implies a dedicated team of highly skilled engineers are required to maintain and optimize such a self-hosted solution. The continuous migration efforts, managing dozens of tenants, and the constant balancing act of performance and cost trade-offs present a non-trivial operational challenge. For organizations without Shopify's engineering depth and resources, the operational overhead of self-hosting could outweigh the benefits, making managed services a more pragmatic choice. The article also hints at the future expansion into AI agents and merchant-facing analytics, which, while promising, will require continued innovation and careful architecture to maintain performance and scalability.

Key Points

• Shopify built its own unified observability platform, 'Observe', on ClickHouse, consolidating metrics, logs, traces, and exceptions.
• The platform handles 100 million events per second (110 GB/s) during peak Black Friday-Cyber Monday traffic.
• ClickHouse delivered a 16x to 30x query performance improvement out-of-the-box, significantly reducing costs compared to fragmented vendors.
• Key technical strategies include buffering in Kafka for large synchronous writes, using materialized views for flexible schemas and correlation, and optimizing schema evolution.
• While self-hosting provided control, Shopify now considers ClickHouse Cloud for faster time-to-value and reduced operational burden.

---

📖 Source: Shopify powers observability for global-scale commerce with ClickHouse