Expedia's STAR: AI Accelerates Incident Response

AI-Powered Observability in Action

Expedia's Service Telemetry Analyzer (STAR) represents a pragmatic and valuable application of AI in the critical domain of production incident response. The platform's strength lies in its focused approach: leveraging LLMs not for autonomous action, but to augment human engineers by synthesizing complex telemetry data into structured root cause assessments. This human-in-the-loop design is crucial for high-stakes environments where validation and ultimate decision-making must remain with experienced personnel. The reliance on predefined diagnostic workflows and domain-specific prompts, rather than more experimental autonomous agent capabilities, ensures consistency and predictability, which are paramount for reliable incident management. The integration with existing observability tools like Datadog and the use of common infrastructure telemetry (Kubernetes, JVM metrics) make its adoption path clear for organizations with similar tech stacks. The asynchronous architecture with Celery and Redis is a sound engineering choice for handling I/O-bound operations and rate limits, demonstrating a well-thought-out implementation. The stated goal of minimizing Time To Know (TTK) and Time To Recover (TTR) is directly addressed by the system's ability to quickly surface potential root causes, thereby reducing the cognitive load and manual effort for engineers during stressful incidents. The planned enhancements, such as service dependency mapping and conversational interfaces, indicate a commitment to continuous improvement and broader utility. The evaluation of STAR within chaos engineering practices is particularly forward-thinking, suggesting its potential to proactively identify vulnerabilities.

However, while the article highlights the benefits, it's important to consider the inherent limitations of relying on LLMs for diagnostic tasks. The accuracy and completeness of the generated root cause assessments are heavily dependent on the quality of the input telemetry and the sophistication of the prompts. The system's current exclusion of advanced LLM features like function calling, RAG, and memory might limit its ability to perform more complex, multi-hop reasoning or to dynamically adapt its analysis based on evolving incident context. While consistency is a benefit, it could also mean missing novel or emergent issues that fall outside the predefined workflows. The success of STAR will ultimately hinge on the continuous refinement of its diagnostic models and prompts, as well as the engineering team's ability to effectively manage the LLM provider integrations and potential drift in model behavior. Furthermore, the article doesn't deeply explore the cost implications of extensive LLM usage for telemetry analysis, which could be a significant factor for large-scale deployments. The dependency on specific telemetry sources also means that services not emitting standardized metrics might not be fully covered by STAR's analysis.

Key Points

• Expedia's Service Telemetry Analyzer (STAR) is an AI-assisted observability platform.
• STAR analyzes service telemetry to accelerate production incident investigation by generating structured root cause assessments.
• The platform uses LLMs in conjunction with predefined diagnostic workflows, keeping humans responsible for validation.
• It integrates with Datadog for metrics and an internal generative AI gateway for LLM access.
• STAR focuses on standardized infrastructure telemetry from Kubernetes and JVM applications.
• The system employs prompt chaining for specialized analyses and consolidates findings into reports.
• Expedia aims to minimize Time To Know (TTK) and Time To Recover (TTR) with STAR.
• The architecture evolved to an asynchronous Celery-based system for better concurrency and rate limit accommodation.
• Current limitations include not using advanced LLM features like function calling or RAG.
• Future plans include incorporating service dependency information and conversational interfaces.
• STAR is being evaluated for use in chaos engineering practices.

---

📖 Source: Expedia Uses AI Driven Service Telemetry Analyzer to Accelerate Incident Investigation