Atlassian's Forge Billing: Scaling Distributed Usage Tracking

Forge Billing: Mastering Scale and Accuracy

Atlassian's Forge billing architecture represents a sophisticated solution to the perennial challenge of accurately tracking and billing for distributed, high-volume usage in a SaaS environment. The emphasis on a centralized ingestion and streaming layer, underpinned by Kafka, combined with the Usage Tracking Service (UTS) as the 'nervous system,' highlights a robust approach to managing complex event flows. The design prioritizes key attributes like schema consistency, reliable delivery, idempotent event processing, and accurate tenant attribution. This is crucial for financial correctness and developer trust. The architecture's ability to handle out-of-order and duplicate events through windowed processing and time-based aggregation demonstrates a mature understanding of real-world distributed system complexities. The layered storage approach, balancing auditability with low-latency analytics, is also a sound practice for modern data-intensive platforms.

However, the article, while technically detailed, could benefit from a deeper dive into the specific challenges encountered during the evolution from a simple extension model to a usage-driven ecosystem. For instance, the 'key complexity' of attribution and shaping, while mentioned, could be elaborated with concrete examples of how these were resolved. Furthermore, while Kafka is mentioned, details on its specific configuration for this scale, including throughput expectations and latency targets, would add significant value. The article also touches on 'internal delivery abstractions,' which, if explained further, could provide insights into Atlassian's specific middleware choices. The reliance on a centralized streaming layer, while providing control, might introduce a potential single point of failure or bottleneck if not meticulously engineered for extreme resilience and scalability. The 'near real-time visibility' for developers is a critical feature, and understanding the trade-offs made to achieve this, especially concerning cost and complexity, would be beneficial. Finally, while the article focuses on the internal architecture, a brief mention of the impact on FinOps practices and the tools or dashboards used for monitoring and cost allocation would round out the picture for readers interested in operational efficiency.

Key Points

• Forge Billing addresses the challenge of tracking and billing for distributed usage in Atlassian's serverless extensibility platform.
• The architecture relies on a centralized ingestion and streaming layer (Kafka-based) and a Usage Tracking Service (UTS) for validation, normalization, and attribution.
• Key design principles include schema consistency, reliable event delivery, idempotent processing, and accurate tenant context attribution.
• The system uses windowed processing and time-based aggregation to handle out-of-order and duplicate events, ensuring financial correctness.
• Layered storage provides immutability for audits and a low-latency layer for analytics and dashboards.
• The platform enables scalable, transparent billing with strong correctness and traceability guarantees for developers.

---

📖 Source: Inside Atlassian’s Forge Billing Architecture for Distributed Usage Tracking at Scale