Open infrastructure for building cloud software factories

Joined June 2021
Warp retweeted
Building Evals on Production Data: 🦄 AI That Works #79 xitter.cf/i/broadcasts/1oJMvNZWv…
7
8
3
110
17,362
At @aiDotEngineer, I presented our skill called /skill-doctor. It's a skill to improve your agent skills. It aggregates your past agent sessions, grades their quality, and propose improvements to fix common failures.
12
10
117
6,020
Getting software factories working is going to be one of the more interesting software engineering challenges in the next few years. Software engineering is becoming factory engineering. Organizations that can make their factories robust, dependable, and self-improving will be able to ship more at better cost and have a competitive advantage.
16
6
1
149
6,610
An interesting engineering challenge this next year will be closing the development loop we've started. We have small loops already (code review, automatic CI fixes, computer use verification), but we need to close the larger loop: let agents suggest what to build next.
I've been thinking about what a true, closed-loop software factory will look like. One where agents know what to fix, what costs to optimize, and the big one: what features to actually build. We're not there yet. But maybe we're getting close...
5
2
39
4,370
Software factories can be a real culture shift. The biggest change for us was moving from individual agents on everyone's machines to one central agent setup (a factory) that we can build as a team. I wrote some learnings on how to navigate that.
40
18
3
258
19,179
We've found a few key pieces to successful software factories, both from our internal testing and from helping companies deploy their own: Factories-as-code: you should configure every part of the factory (agent orchestration, automations, permissions, sandbox configuration, etc) as code. This enables testing of different factory configurations to see which are most efficient, highest quality, etc. Multi-model & multi-harness: you should make sure that your factories are able to use the latest models, both frontier and open-weight, and use different coding agent harnesses like Claude Code and Codex. Data ownership: you should make sure you store and own all data coming out of your factory. This is the raw material for improving its operations.
56
18
8
207
14,990
Someone asked how I defined a software factory recently, and I think my definition is different from others. To me, a software factory is centralizing your system of agents into one place to build software. That means going from custom coding setups on everyone's machines to a single definition. In practice, it's one source of truth for agent automations, repository access, secret access, sandbox environments, model selections... every aspect of the stack. I think loops and automations are a secondary aspect of this that gets unlocked once you unify everything into an "always-on" agent system. But the heart of it is having the whole team agree on the agent setup, the same way we agree on CI, lint rules, and all the automations we had pre-AI. This also makes it clear what an engineer's job is now: to make the agents better at building software. Not to be prompt monkeys! We like to call it "factory engineering"
38
24
6
230
27,027
I chatted with @hudaman (Sr Engineering Manager, Uber) and @cmhandel (Director of Product, AI Platform, Faire) to compare notes on adopting software factories. We discussed: - The best starting point for factories - Code review and spec writing - How to navigate team culture changes And how everyone's handling software factory adoption at their companies. Many different approaches to this youtu.be/0gDHnH89aQs
3
5
1
33
7,198
My @aiDotEngineer SF talk is on YouTube. My take is that software engineering is becoming factory engineering: building the system of agents that builds the product. We used the Warp open source repo (now at 65k GH Stars) as a testbed for this idea: youtube.com/watch?v=tUPPVhBB… 🔖
8
7
82
7,457
Warp retweeted
Day 2 at @aiDotEngineer NYC Come find us to see what the Warp team’s been up to over the last couple months! and plushies 🙂
1
32
1,885
I'll be joining @sergical at @sentry on October 22nd to talk about connecting error monitors to coding agents! It's how we've closed the loop in our own software factory to have agents assign themselves work automatically. The flow we're using right now: - Connect to an alert webhook like Sentry - When errors happen, start a cloud agent run with the alert and the repo as context - Have the agent ping humans for code review (or better: with an HTML summary of the problem and the solution) - Merge and keep monitoring This closes the loop on reducing software bugs. We'll show you how to deploy it in-code yourself!
13
3
3
30
6,194
Got to spend some time back in the Azure Portal while building this out...very nostalgic! Check it out -- I'm particularly happy with the automations support we landed here. Warp is one of the few (only?) cloud agent platforms that lets you integrate with ADO's events so easily.
Warp Factories now supports Microsoft Teams and Azure DevOps. Software factories should be built on flexible infrastructure to run agents from anywhere. Thats why we’re the first in the industry to support the full Microsoft suite across software factory workflows.
3
1
16
1,892
The automation loop driving our software factory. It's our default for anything built with Warp Factories: triage, spec, implement, review
Here's the multi-agent system driving our software factory right now. Core pieces: - We say what to build (sometimes Linear issues, mostly Slack threads) - Triage agent researches and decides whether to implement or to ask questions (think /grill-me but smarter) - Implementation agent builds, maybe collaborating with us on a spec with a subagent first - Verification subagent tests the implementation agent's work e2e - Code review agent cycles with the implementation agent for a few rounds before human review - Human reviews and ships - Monitoring automation responds to alerts and files issues (our first step to closing the factory loop) There's room to simplify this. We're benchmarking whether we need separate agents for steps like triage vs. implementation, or if a single high-intelligence model can do most of it. The core loop remains the same though. Curious how this compares to factories others are deploying
9
7
1
189
27,088
Here's the multi-agent system driving our software factory right now. Core pieces: - We say what to build (sometimes Linear issues, mostly Slack threads) - Triage agent researches and decides whether to implement or to ask questions (think /grill-me but smarter) - Implementation agent builds, maybe collaborating with us on a spec with a subagent first - Verification subagent tests the implementation agent's work e2e - Code review agent cycles with the implementation agent for a few rounds before human review - Human reviews and ships - Monitoring automation responds to alerts and files issues (our first step to closing the factory loop) There's room to simplify this. We're benchmarking whether we need separate agents for steps like triage vs. implementation, or if a single high-intelligence model can do most of it. The core loop remains the same though. Curious how this compares to factories others are deploying
128
80
18
1,403
152,444
Full rundown of our software factories platform if you've been curious
This past month, we shipped Warp Factories. It's a really robust, flexible solution to deploy software factories with any model + harness configuration. There's a lot of moving parts, so I recorded a complete walkthrough of what it can do. Hope it's helpful!
3
1
63
14,669
You should benchmark models against your own conversations to find the right one from a cost/performance perspective. Here's the benchmarking setup we use:
5
3
59
107,589
Software engineering is shifting to "factory engineering:" the focus isn't just on building the product, but improving throughput. One measure of factory efficiency is "human touches per pr." Over time you want to drive this down.
13
10
1
96
106,264
Agentic engineering is really two disciplines now: - Product engineering: ensuring the quality and usefulness of product - Factory engineering: deploying agents to build that product, optimizing system efficiency and output quality Here's how to adapt:
41
36
9
311
25,153
Warp retweeted
We @warpdotdev are teaming up with @warpdotco to host a poker tournament this Wednesday evening in NYC! Luma link & buy-in details in thread 😁
11
3
3
24
6,779