Datadog's AI-Powered Migration: Claude & Cursor in Action

AI-Assisted Refactoring for Performance

Datadog's experience with Claude and Cursor for migrating their Stream Router system offers a compelling case study in leveraging AI for complex refactoring tasks. The core innovation lies in their systematic, test-driven approach, where AI acted as an accelerator rather than an autonomous code generator. By providing clear inputs – old implementation, new schema, and failing tests – they effectively guided the AI, using the comprehensive test suite as the ultimate arbiter of correctness. This methodology mitigates many concerns about AI-generated code reliability and demonstrates a pragmatic path to integrating LLMs into critical development workflows.

The article highlights the crucial role of strong code modularity and a robust testing infrastructure. These foundational elements were not replaced by AI but were essential enablers for its successful application. The parallel infrastructure and blue/green deployment strategy further underscore a mature DevOps practice, minimizing risk during a high-stakes migration. The dramatic improvements in performance (45 minutes down to seconds) and cost reduction (90% database cost savings) are tangible outcomes that validate this approach. However, the limitations noted, such as the AI's inability to discover niche optimizations independently and its tendency to generate suboptimal queries without explicit guidance, are important reminders that AI is a tool that requires skilled human direction and validation. The high token consumption also points to areas for future optimization in prompt engineering and data handling for LLMs.

This case study is highly relevant for organizations facing similar challenges with legacy systems, particularly those with complex data relationships and performance bottlenecks. Developers and engineering leads can draw direct inspiration from Datadog's methodical integration of AI, emphasizing the need for well-defined inputs, rigorous testing, and a phased rollout. The success hinges on augmenting human expertise with AI capabilities, rather than replacing it. The lessons learned about prompt engineering and the iterative refinement process are invaluable for anyone looking to adopt AI in their development pipeline.

Key Points

• Datadog used Claude and Cursor to accelerate a systematic, test-driven refactoring process for migrating their Stream Router system.
• The migration involved redesigning the schema to reflect relationships and refactoring code from a key-value model to a relational PostgreSQL backend.
• Key enablers for success included strong code modularity, a comprehensive test suite acting as a pass/fail criterion, and a parallel infrastructure for side-by-side comparison.
• The migration process was phased: understanding function intent with AI, fixing failing tests with focused prompts, and deploying via a blue/green approach.
• Limitations included AI's difficulty with niche optimizations and suboptimal query generation without explicit human input, as well as high token consumption.
• The migration resulted in dramatic performance improvements (operations from 45 mins to seconds), significant cost reductions (90% database cost), and reduced data storage.

---

📖 Source: How Datadog Used Claude and Cursor for Test-Driven Production Migration