Spotify's Honk: AI Rewrites Codebase Continuously

AI's Fleet Management Revolution

Spotify's 'Honk' project represents a significant leap forward in tackling the perennial 'maintenance problem' in software engineering. The core innovation lies in leveraging LLMs not just for code generation, but for an end-to-end, fleet-wide migration process. By abstracting away the complexities of diverse build systems and testing environments through a unified 'verify' tool and intelligent failure summarization, Honk effectively democratizes the ability to perform large-scale, automated code refactoring. The journey highlights the critical importance of decoupling AI agent execution from CI verification runtimes and the evolution of feedback loops, moving from basic LLM-as-a-judge to relying on robust verification steps within prompts. This approach addresses the limitations of traditional, script-based fleet management, which struggles with the 'long tail' of complex, edge-case migrations.

Key Points

• Spotify faces the 'maintenance problem,' where developers spend less than an hour coding daily due to dependency updates, version migrations, and framework changes.
• Traditional fleet management uses scripts to automate migrations across thousands of codebases, significantly reducing adoption time from a year to under a week, but struggles with complex edge cases.
• Honk, an AI coding agent, was developed to overcome the limitations of scripts by leveraging LLMs for more versatile code rewriting and handling complex scenarios.
• Key architectural insights include decoupling CI verification runtimes from AI agents and developing a unified 'verify' tool that abstracts diverse build systems (Maven, Yarn, Bazel).
• LLMs are used for summarizing build failures, reducing verbose output to actionable insights for the agent.
• The evolution from 'LLM-as-a-judge' to integrating verification steps directly into prompts reflects the increasing capability of LLMs in self-correction and adherence to requirements.
• A major challenge was the discrepancy between local development environments and CI/CD infrastructure (e.g., permissions, Docker support, OS differences), leading to infrastructure-related failures that needed to be addressed to ensure migration correctness.

---

📖 Source: Presentation: Rewriting All of Spotify's Code Base, All the Time