Deep Research Agents: Production Lessons

Beyond Basic RAG: Agentic Synthesis

Sarang Kulkarni's presentation offers a compelling look into the challenges and sophisticated solutions for building "Deep Research Agents," particularly in high-stakes industries like pharmaceuticals. The core innovation lies in the evolution from simple RAG to an "Agentic RAG++" system, incorporating structured loops for clarification, research (think, plan, execute, reflect, adjust), and writing. The emphasis on agentic loops, harness engineering, and explicit think-act cycles for long-horizon tasks is highly relevant. The detailed breakdown of the RAG tool's components (weighted hybrid search, re-ranking) and the text2sql tool's feedback mechanism highlights practical engineering considerations. The identification of issues like context anxiety and poor self-evaluation due to incomplete data, and the proposed solutions like reflection loops, are crucial for anyone attempting to build robust AI agents.

However, while the article details the 'what' and 'how' of their agentic system, it could benefit from a more quantitative assessment of performance improvements gained through these enhancements. For instance, how much did latency decrease, or how did the accuracy of generated reports improve compared to earlier RAG versions? The discussion on harness engineering is promising, shifting focus from prompt engineering to automated task execution, but a deeper dive into specific harness components and their impact on reliability and accountability would be valuable. The potential for these agents to democratize complex research by making fragmented knowledge accessible is immense, benefiting researchers, analysts, and decision-makers across scientific and industrial domains. The technical implications are significant, pushing the boundaries of what LLMs can achieve beyond simple retrieval and generation towards true analytical reasoning and synthesis. The comparison to existing solutions is implicitly made by highlighting the limitations of basic RAG for complex tasks, positioning Agentic RAG++ as a more advanced paradigm.

Key Points

• Deep Research Agents are AI systems designed for multi-step internet research, complex task execution, and structured analytical report generation, going beyond simple Q&A.
• The evolution from basic RAG to "Agentic RAG++" is crucial for complex research, incorporating structured loops for clarification, research, and writing.
• Key components of the Agentic RAG++ system include clarification, research (think, plan, execute, reflect, adjust), and writing loops.
• Practical challenges like token costs, high latency, context anxiety, and incomplete data leading to poor self-evaluation must be addressed.
• Techniques like explicit think-act loops, reflection steps (data and process), and harness engineering are vital for building reliable and accountable AI agents.
• Harness engineering focuses on designing tools, memory systems, validation checks, constraints, and feedback loops to enable automated task execution by AI agents, shifting focus from prompt engineering.

---

📖 Source: Sarang Kulkarni on Lessons from Building Deep Research Agents in Production