Pinned Tweet
Six years ago, DBOS started as a joint research project at MIT and Stanford on applying database principles to modern applications. Today we announced a major milestone: the issuance of U.S. Patent for our database-centric durable workflow architecture.
This recognizes the unique approach we’ve taken to simplify durable workflow orchestration by turning the database itself into a workflow engine, rather than hosting specialized infrastructure to do the same. We're so grateful to be building this!
DBOS retweeted
I always found Webflow hard to navigate (it was a choice from 3 years ago). So my team just migrated the entire DBOS site from Webflow to React, all done by Claude Code. The agent worked on its own for three hours and rebuilt the whole site while preserving the original design.
Now the entire website is just code, and I can finally change anything I want.
New releases across DBOS Transact SDKs: TypeScript v5.3, Python v3.3, and Go v1.7.
One important change across all three releases: we moved the DBOS Conductor client into a separate Enterprise package. This change will allow us to deliver several exciting new features in the coming weeks.
If you're only using the open-source DBOS libraries, nothing changes and just upgrade as usual.
If you're using DBOS Conductor or DBOS Cloud, you'll need to install the Enterprise dependency when upgrading:
TypeScript:
𝗇𝗉𝗆 𝗂 @𝖽𝖻𝗈𝗌-𝗂𝗇𝖼/𝖽𝖻𝗈𝗌-𝖾𝗇𝗍𝖾𝗋𝗉𝗋𝗂𝗌𝖾
Python:
𝗉𝗂𝗉 𝗂𝗇𝗌𝗍𝖺𝗅𝗅 𝖽𝖻𝗈𝗌[𝖾𝗇𝗍𝖾𝗋𝗉𝗋𝗂𝗌𝖾]
Go:
𝗂𝗆𝗉𝗈𝗋𝗍 _ "𝗀𝗈.𝖽𝖻𝗈𝗌.𝖽𝖾𝗏/𝖽𝖻𝗈𝗌-𝖾𝗇𝗍𝖾𝗋𝗉𝗋𝗂𝗌𝖾-𝗀𝗈"
We'll be rolling out the same change for Java soon.
Full release notes:
TypeScript: github.com/dbos-inc/dbos-tra…
Python: github.com/dbos-inc/dbos-tra…
Go: github.com/dbos-inc/dbos-tra…
DBOS retweeted
AI is pushing us to rethink how we build applications. Here's a new blog post from Mike Stonebraker. He argues that applications should move toward durable workflows as AI agents become more common and complex.
A lot of developers are still stitching together stateless services, queues, databases, and state machines to keep things running. That was already painful for traditional applications. Now it's even worse for AI agents that execute long-running tasks, call external tools, and perform actions with side effects.
Mike's argument is that durable workflows should be the default, and you should use your database as the orchestrator.
Join us for the next DBOS User Group Meetup on October 22!
This month, @Credix_finance and Cortex will share how they're using DBOS to build durable workflows for commercial lending and AI software factories. We'll also share the results of our roadmap survey and open the floor for discussion.
As always, there will be plenty of time to connect with fellow DBOS users and our engineering team. We'd love to hear your feedback and ideas!
Register here: luma.com/2f5n3nr1?tk=GZ8SFs
DBOS retweeted
DBOS just hit 4M monthly open-source downloads
1 month after hitting 3M, 4 months after hitting 1M
Building durable execution on Postgres, with no other infrastructure required, is fundamentally the right architecture.
Let’s keep growing!
DBOS retweeted
It's interesting to see how differently AI and humans fix bugs. We recently got a bug report about cron scheduling during daylight saving transitions. We received two AI-generated PRs trying to fix it by patching the vendored croniter code.
But the real fix was to simply update the croniter dependency, which already included the upstream fixes.
I think today's AI is pretty good at patching code to fix an immediate problem. But the next step is learning to see the bigger picture, like understanding the root cause and finding the simplest/most maintainable solution.
DBOS retweeted
Cool new feature: you can now “rewind” your workflows to any previous step.
For example, if your workflow failed because of a bug or outage, once you resolve the issue, you can rewind all affected workflows to the step before the issue happened. That way, they can recover from just before the failure instead of restarting from the beginning.
You can rewind workflows programmatically, from a script, or manually, from your dashboard.
DBOS retweeted
Interesting to see how Dosu corrects their agent memory and make sure it says DBOS :)
Does your coding agent still recommend a tool your team replaced? We found a Celery migration comment that survived the move to DBOS and the removal of Celery. Semantic memory needs to preserve when a fact no longer applies. dosu.dev/blog/semantic-memor…
DBOS retweeted
Does your coding agent still recommend a tool your team replaced? We found a Celery migration comment that survived the move to DBOS and the removal of Celery. Semantic memory needs to preserve when a fact no longer applies. dosu.dev/blog/semantic-memor…
Hot on the heels of last's weeks v1.1 release, today we released v1.2 DBOS Java SDK.
This release completes a pair of changes (workflow payload tables and delay debouncer) that stated in v1.1. By splitting the work across two minor releases, we avoided making a breaking change.
We also changed most of the timestamps to use the database clock instead of the executors and added the batched partition dequeue optimization when partition concurrency is one.
Release notes: github.com/dbos-inc/dbos-tra…
DBOS retweeted
Let’s talk about zombies!
Zombies are something you have to deal with building any distributed system, but in our case Postgres-backed durable workflows. Here’s the scenario:
You have a process happily executing a workflow. It chugs along for a while, then goes dark. Your system doesn’t know what happened (crash, network partition…) but after some grace period, it needs to recover the workflow to a healthy process so it keeps executing.
But then, the original process comes back! It’s a “zombie”, a process come back from the dead. Now, two processes are executing the same workflow at the same time. This is a dangerous situation: the best case is they’ll duplicate all work, the worst case is they’ll take different steps and produce divergent workflow histories.
So what do you do?
The solution we built is workflow ownership. Whenever a process starts executing a workflow it generates a UUID and “stamps” the workflow’s row in Postgres with that UUID. For as long as the workflow is stamped with that UUID, the generating process “owns” the workflow and can update its history.
This notion of ownership lets us easily handle zombies. Whenever we apply a control-plane operation to a workflow, like recovering it to another process or cancelling it, we NULL out its stamp in Postgres. If a new process later starts executing, it sets a new stamp. When the zombie comes back, it sees it no longer owns the workflow and safely halts execution, deferring to the new owner.
What’s the takeaway? When building distributed systems, you can’t assume processes always cleanly fail. You have to have a plan for dead processes coming back and making sure those zombies can’t corrupt data.
DBOS retweeted
The harvest season is here, and we've shipped a bunch of things at DBOS over the past month. I put together a new blog post covering what's new, including:
- 10x faster data retention
- Python v3.0 & TypeScript v5.0
- Rust SDK preview
- Fair queueing
- Autoscaling policies
- @vercel AI SDK integration
- Run in @supabase Edge Functions
...and much more.
Just released DBOS Python 3.2 and TypeScript 5.2.
These releases include several improvements to datasources:
- Programmatic datasource migrations
- Garbage collection for datasource checkpoints
- Custom sessionmaker support for the Python SQLAlchemy datasource
Under the hood, we also added a new workflow ownership system that helps DBOS detect duplicate workflow executions more reliably, for example, executions caused by "zombie" processes.
TypeScript release notes:
github.com/dbos-inc/dbos-tra…
Python release notes:
github.com/dbos-inc/dbos-tra…
DBOS retweeted
Cool new feature: workflow ID reuse policies
You can start a durable workflow with a set workflow ID, which acts as an idempotency key to guard against duplicate workflow executions.
By default, if you start multiple workflows with the same ID, the later executions “attach” to the original.
However, sometimes, you want to instead detect the duplication and take a different action, like starting a new workflow. Reuse policies let you do that: you can set a reuse policy of “reject” to atomically reject duplicate workflow IDs with an error, then catch that error to take a different action.
DBOS retweeted
Separating workflow inputs/outputs into their own tables laid the groundwork for our batching and data locality improvements in data retention (see our recent blog post).
One note: we deliberately left out foreign key references between those tables, so we can delete from each table in batches without the overhead of FK checks or cascading deletes. Sometimes what you don't see in the schema can matter for performance.
Optimization story time!
So in DBOS, every workflow is represented by a row in a Postgres workflow status table, which stores the workflow’s status. I was trying to figure out where to store the workflow’s inputs and outputs.
The natural thing to do, and the thing we did originally, was to make them columns of the workflow status table. That way, they’re updated with the rest of the status.
The issue is that workflow inputs and outputs can be large. This is tricky because Postgres does multi-version concurrency control (MVCC), meaning that every time you update any value in a row, Postgres rewrites the entire row. So storing inputs and outputs directly in the workflow status table is dangerous, because they could get rewritten many times by unrelated status changes.
But this isn’t the end of the story, because Postgres has another feature specifically to manage large columns: aptly named “the oversized-attribute storage technique (TOAST)”. What this does is store large column values in their own separate “TOAST tables” both to give them more space and to make sure they aren’t rewritten every time the main row is updated. So storing inputs and outputs in the workflow status table is safe, because Postgres TOASTs them and they aren’t rewritten.
But this also isn’t the end of the story, because we need to be able to efficiently delete workflows after they’re done processing. And if workflow inputs and outputs are TOASTed, they’ll be stored in random locations on disk far from their parent workflow status row. So that means when we do a batch delete of workflows, we have to seek all their TOASTed inputs and outputs, which causes cache thrashing and terrible performance. So storing inputs and outputs directly in the workflow status table is dangerous, because it makes deleting workflows too expensive.
So what we finally ended up doing is normalizing the main workflow status table and pulling workflow inputs and outputs into their own tables. When deleting workflows, we delete from each table separately, so all data is deleted in insertion order, giving us good cache locality and performance.
What’s the takeaway from all this? At enough scale, all abstractions are leaky! None of these details (MVCC, TOAST, cache locality) are in the SQL spec, but you need to understand them to make a Postgres-backed system fast. Ultimately, making any system work well requires deeply understanding how it interacts with everything below it.
DBOS retweeted
Optimization story time!
So in DBOS, every workflow is represented by a row in a Postgres workflow status table, which stores the workflow’s status. I was trying to figure out where to store the workflow’s inputs and outputs.
The natural thing to do, and the thing we did originally, was to make them columns of the workflow status table. That way, they’re updated with the rest of the status.
The issue is that workflow inputs and outputs can be large. This is tricky because Postgres does multi-version concurrency control (MVCC), meaning that every time you update any value in a row, Postgres rewrites the entire row. So storing inputs and outputs directly in the workflow status table is dangerous, because they could get rewritten many times by unrelated status changes.
But this isn’t the end of the story, because Postgres has another feature specifically to manage large columns: aptly named “the oversized-attribute storage technique (TOAST)”. What this does is store large column values in their own separate “TOAST tables” both to give them more space and to make sure they aren’t rewritten every time the main row is updated. So storing inputs and outputs in the workflow status table is safe, because Postgres TOASTs them and they aren’t rewritten.
But this also isn’t the end of the story, because we need to be able to efficiently delete workflows after they’re done processing. And if workflow inputs and outputs are TOASTed, they’ll be stored in random locations on disk far from their parent workflow status row. So that means when we do a batch delete of workflows, we have to seek all their TOASTed inputs and outputs, which causes cache thrashing and terrible performance. So storing inputs and outputs directly in the workflow status table is dangerous, because it makes deleting workflows too expensive.
So what we finally ended up doing is normalizing the main workflow status table and pulling workflow inputs and outputs into their own tables. When deleting workflows, we delete from each table separately, so all data is deleted in insertion order, giving us good cache locality and performance.
What’s the takeaway from all this? At enough scale, all abstractions are leaky! None of these details (MVCC, TOAST, cache locality) are in the SQL spec, but you need to understand them to make a Postgres-backed system fast. Ultimately, making any system work well requires deeply understanding how it interacts with everything below it.