Microsoft Boosts PostgreSQL with In-DB Durable Workflows

Database as a Workflow Engine

Microsoft's open-sourcing of pg_durable represents a compelling evolution in how we can architect data-intensive applications. The core innovation lies in bringing durable execution, previously the domain of specialized orchestration platforms, directly into PostgreSQL. This promises to significantly simplify development by eliminating the need to manage complex external systems for retries, state management, and fault tolerance. By allowing workflows to be defined and executed natively within SQL, developers can leverage their existing database expertise and infrastructure to build more robust and resilient long-running processes. This approach has the potential to drastically reduce operational overhead and the cognitive load associated with distributed system development, especially for tasks like ETL, data pipelines, and even agent-based computations.

Key Points

• Microsoft has open-sourced pg_durable, a PostgreSQL extension for in-database durable workflow execution.
• It allows long-running, fault-tolerant SQL functions to run natively within PostgreSQL, eliminating external orchestration systems.
• Workflows are defined in SQL using a DSL with operators for sequential and parallel execution, retries, and checkpointing.
• Execution state is persisted in PostgreSQL tables, enabling automatic resumption from failure points.
• Key use cases include vector embedding pipelines, scheduled maintenance, and workflows depending on external APIs.
• The extension is architecturally minimal, consisting of a PostgreSQL extension and a background worker built on Rust libraries (duroxide, duroxide-pg).

---

📖 Source: Microsoft Open-Sources PostgreSQL Extension for In-Database Durable Execution