HubSpot's 20B Vector Scale: VaaS Deep Dive

Beyond the Database: Operationalizing Vector Search at Scale

HubSpot's journey to scaling semantic search to 20 billion vectors is a compelling case study in building robust, production-ready AI infrastructure. The decision to build VaaS (Vector as a Service) as an abstraction layer over Qdrant highlights a critical trend: the database is only one piece of the puzzle. The real challenge lies in the surrounding automation, data management, and operational tooling required to support such massive scale. Their move from Helm to a custom Kubernetes Operator framework is particularly noteworthy, demonstrating the limitations of declarative approaches for complex, stateful systems that demand dynamic scaling and self-healing capabilities. The emphasis on Translators for reconciling desired and actual system states showcases a mature approach to managing distributed systems, reducing manual toil and improving resilience. The article effectively conveys that achieving high retrieval quality and low latency at scale is as much an operational feat as it is a retrieval algorithm problem.

However, while the article details the 'how' of their scaling, it could benefit from a deeper dive into the 'why' behind certain architectural choices, especially concerning potential trade-offs. For instance, the decision to run Qdrant in-house, while offering granular control, introduces significant operational overhead compared to managed services. The article touches upon cost controls like quantization and on-disk storage, but a more detailed discussion on the economic implications of managing such a large vector database internally versus leveraging cloud-native vector databases or managed Qdrant instances would be valuable for organizations considering similar paths. Furthermore, while mentioning agent usage and RAG, a more specific exploration of how the vector search infrastructure directly impacts the performance and user experience of these AI applications would add further depth. The article's focus on operational maturity is excellent, but understanding the specific performance metrics and user-facing improvements directly attributable to the VaaS platform would provide a more complete picture of its impact.

Key Points

• HubSpot built VaaS (Vector as a Service) as an abstraction layer over Qdrant to manage 20 billion vectors across 38+ teams.
• The platform provides access control, embeddings generation, data versioning, and feedback collection, enhancing Qdrant's capabilities.
• Key Qdrant features leveraged include on-premises deployment, named vectors, hybrid search, multi-stage querying, and cost controls.
• Manual management proved unsustainable; HubSpot transitioned to an internal Kubernetes Operator framework for automated cluster management, scaling, and recovery.
• The use of 'Translators' reconciles desired and actual system states, automating cluster operations and reducing operational load.
• Scaling vector search is as much about operational maturity, automation, and infrastructure as it is about retrieval quality and latency.
• The trend towards simplifying retrieval systems while maintaining performance is evident across the vector search market.

---

📖 Source: How HubSpot Scaled Semantic Search to 20 Billion Vectors