The Lede

Durable workflows are a crucial component of modern software systems, ensuring that tasks are completed reliably even in the face of failures or crashes. Researchers and developers have long turned to specialized systems like Temporal and Restate to build these workflows. However, a growing number of experts are discovering that PostgreSQL, the popular open-source database, provides a powerful tool for building durable workflows. By leveraging PostgreSQL's multi-version concurrency control and queueing capabilities, developers can create reliable systems without sacrificing performance.

Background & Context

Durable workflows have been a focus of research and development in the software industry for several years. These systems enable long-lived, reliable functions to run despite crashes, restarts, and network failures. PostgreSQL's features make it an attractive choice for building durable workflows. The database's multi-version concurrency control (MVCC) allows for efficient and scalable queueing, while its state checkpoints provide a robust way to store and recover workflow state.

Deep Dive

One of the key reasons PostgreSQL is well-suited for durable workflows is its MVCC. This feature enables multiple versions of data to be stored simultaneously, allowing for efficient and scalable queueing. By using PostgreSQL's `SELECT ... FOR UPDATE` and `SELECT ... FOR UPDATE SKIP LOCKED` clauses, developers can build robust and scalable distributed queues. Additionally, PostgreSQL's state checkpoints provide a robust way to store and recover workflow state. This allows developers to recover workflows from their checkpoints in case of server crashes or failures.

Expert Angle

According to Peter Kraft, a developer who has extensively used PostgreSQL for durable workflows, 'PostgreSQL is incredibly powerful, with many features that make it easier to build performant and featureful durable workflows.' Armin Ronacher, a well-known expert in durable execution, agrees, stating that 'PostgreSQL is excellent at queues thanks to `SELECT ... FOR UPDATE SKIP LOCKED`, and because it's a database, you can also use it to store the state.' While some experts argue that specialized systems like Temporal and Restate are still necessary for complex workflows, many agree that PostgreSQL provides a powerful tool for building reliable systems.

What Comes Next

As the use of durable workflows continues to grow, PostgreSQL's popularity is likely to increase. Developers are already beginning to explore the use of PostgreSQL for building durable workflows, and the database's features are well-suited for this task. However, it remains to be seen whether PostgreSQL will become the go-to choice for durable workflows, or whether specialized systems will continue to dominate this space.