Pelago's 2-Week Serverless AI Assistant

Event-Driven AI at Speed

The article brilliantly showcases the power of serverless and event-driven architecture for rapid AI solution development in a highly regulated industry. The separation of concerns, leveraging Amazon SNS for fan-out and AWS Lambda for decoupled processing, is a core strength. Pelago's approach to pre-generating AI suggestions asynchronously, ensuring sub-100ms retrieval for the care team, is a masterclass in user experience optimization while handling computationally intensive LLM tasks. The emphasis on maintaining PHI within the AWS VPC using VPC endpoints for Amazon Bedrock is crucial and well-executed, addressing critical compliance needs without public internet exposure. The polyglot approach to Lambda functions (Python for Bedrock, TypeScript for retrieval) is a pragmatic choice that maximizes developer efficiency and leverages the best tool for each job.

However, a deeper dive into the prompt engineering nuances beyond 'empathy and validation' could offer more granular insights. While the article mentions tuning the model to maintain contextual continuity and steer away from dismissive language, specific examples of how this was achieved through prompt structure or guardrails would be valuable. Furthermore, while the 79.6% helpfulness rating is strong, understanding the methodology behind this internal measurement (e.g., coach feedback mechanisms, specific criteria for 'helpful') would add more weight. The article also touches on handling LLM throttling or model unavailability with retries, but elaborating on the retry strategy (e.g., exponential backoff, maximum attempts) would be beneficial for readers facing similar challenges. Despite these minor points, the overall achievement of concept to production in two weeks with significant positive business impact is highly commendable and provides a compelling blueprint.

Key Points

• Pelago built a serverless AI assistant for substance use disorder support in just two weeks.
• The solution uses an event-driven architecture with Amazon SNS and AWS Lambda for decoupled processing.
• Amazon Bedrock is used for LLM-based suggestion generation, with context retrieved from Amazon DynamoDB.
• PHI is secured by keeping all AI integrations within the AWS VPC using VPC endpoints for Amazon Bedrock.
• Pre-generation of suggestions ensures sub-100ms retrieval for care team members, improving user experience.
• The system achieved a 40% reduction in response preparation time and a 79.6% helpfulness rating for AI suggestions.
• Serverless services like Lambda provide organic scaling and eliminate infrastructure management overhead.
• Polyglot Lambda runtimes (Python and TypeScript) were used to leverage the best language for specific tasks.
• DynamoDB is used for high-throughput message storage, while RDS MySQL stores structured assistant suggestions.

---

📖 Source: Building a serverless AI assistant at Pelago: concept to care in two weeks