Brendon Archuleta (@Brendon_Arch) header
Brendon Archuleta (@Brendon_Arch) profile picture

Brendon Archuleta

@Brendon_Arch

Founder notes. AI ops. Building WednesdAI in public.

 Salt Lake City, UT
Wednesday-ai.com
Joined September 2012
  • Tweets2.4K
  • Following458
  • Followers305
Agreed. Ops without receipts is just motion. We log run ID, decision ID, tool result, and policy outcome on every boundary so a live experiment can be replayed later, not reconstructed from vibes.
3
1,228
Brendon Archuleta retweeted
Peter Steinberger 馃 (@steipete) profile picture
Peter Steinberger 馃
@steipeteOct 20
Kinda sad how many of the replies here are now clearly slop. My claw mutes most of these automatically. I wonder what people try to gain from that.
492
31
16
2,090
221,999
Agent traces are going mainstream this week. Collecting them was never the hard part. The hard part is the failure that looks like a success in the trace.
2
31
A trace tells you what the agent did. It doesn't tell you whether that was the job.
14
An agent reviewing its own work in the same session is just the author with a new hat. Fresh context, a different agent, a written pass or fail. Then a human owns the merge.
2
40
One of my bots just clicks through every new deploy and reports what broke. It has no stake in the code being right, which is the whole point.
25
Your ChatGPT referral line fell off a cliff. Demand might not have. When the referrer drops the UTM tags, analytics files the visit as Direct and everyone writes a postmortem about traffic that never left.
1
61
Attribution that depends on a third party remembering a query string is a favor, not a measurement system.
24
Shipping got cheap. Finding out what you actually shipped didn't. So one of my bots doesn't build anything. It clicks through every new deploy and reports what broke before a client sees it.
1
1
44
Passing tests tell you the code did what you wrote. Clicking through the live deploy tells you what a client is about to see. Who on your team does that second part?
20
Most teams can't tell you the ROI of their automation, and it's not because the tools don't work. The tracking stops at the edge of each tool. The person moves across five of them.
1
26
Attribution has to follow the person, not live inside one platform.
15
What actually matters once agents ship: 1. Evals on your real tasks 2. Traces someone actually reads 3. Separate bots checking the work 4. A harness tuned to the job The model is the easy part.
2
41
Most of the debate goes to picking the model. Most of the failures live in 1 through 4.
9
Every time a check kicks back an agent's change, that's a test case you didn't have to write. Keep the rejections and you get a regression suite for free.
1
42
Most teams throw that signal away the moment the fix lands.
20
I run QA bot teams on real client work, bots checking other bots. What do you want to know about running that day to day?
1
35
Failures, costs, what I'd do differently. Ask anything.
1
21
Multi-agent systems rarely fail loudly. One agent drifts, the next one trusts it, and you end up with a confident wrong answer.
1
1
42
That's why the handoffs need their own checks. QA bot teams that verify what got passed along, not just what came out the end.
30
You didn't solve attribution. You moved it. When an agent buys the ticket, nobody clicks anything, and the old tracking goes blind. The hard part was always proving which touchpoint caused the sale.
1
40
Clicks were always a stand-in for intent. Once the buyer is an agent, the proof has to come from the systems that record the sale, not from whoever got the last click.
27
Load more