🚀 Building Agentic AI @ Accordion

Joined August 2019
How do you communicate this to people who haven't seen this yet
All models, no matter how smart, will eventually build systems that they can no longer understand or maintain, if you let them. Fable 5 finally outbuilt itself, and flailed on me for a week. Fable 5.1 looks like it will fix it. For now. But you have to keep an iron grip on system size, or it'll run away from you.
1
57
Chris Ellis retweeted
A story from a startup where the CTO recently resigned, despite having 2% of common shares: "My CEO was nontechnical, and was both moving too slow and too fast with AI. Too slow as in: did not take the time to understand what our customers wanted. We built a TON of stuff, it confused then, they churned, growth stalled, word-of-mouth growth was gone. Too fast as in: deprioritizied core systems reliablity in favor of shipping AI work to prod (which did not have any commercial potential)"
85
115
25
2,362
211,679
Chris Ellis retweeted
"Now I could write at length about all the caveats. All the potential dangers. All the ways it's not quite there yet. But it would be an insult to the marvel of this moment... This is simply the most fun I've ever had with a computer." world.hey.com/dhh/endless-ex…
64
144
34
2,043
173,007
Why do pilots use checklists? Because anything that can be made deterministic, should be made deterministic.
82
213
32
3,050
96,506
Chris Ellis retweeted
Thinking about a workflow like this for helping prevent comprehension debt on a fast-moving repo: Fast-moving repo w/lots of changes -> once per day/week, grab a diff of the changes -> feed it to an LLM to output a podcast transcript, focusing on the 'why' of what changed more than the 'what -> feed the podcast transcript to a TTS API -> publish to a private podcast feed -> listen on your commute/morning run/lunch break I'm doing this for my personal wiki, which ingests from X/Slack/Discord/Gmail, and it's super-efficient for keeping me in the loop.
57
28
4
816
56,226
博客更新: 为什么我用mattpocock/skills替代了superpowers 最近我用@mattpocockuk 的skills替代了之前在用的obra/superpowers,本文是我的一些想法。 justinyan.me/post/6676
38
99
16
741
233,402
Chris Ellis retweeted
“Loop engineering” is a hot buzzphrase after mentions of it by Boris Cherny (Claude Code’s creator) and Peter Steinberger (OpenClaw's creator) went viral on social media. Loops are now a key part of how we get AI agents to iterate at length to build software. In this letter, I’d like to share my 3 key loops, shown in the image below, for building 0-to-1 products. These loops guide not just how I build software, but also how I decide what software to build. Agentic coding loop: Given a product specification and optionally a set of evals (that is, a dataset against which to measure performance), we can have an AI agent write code, test its work, and keep iterating until the code is bug-free and meets its specification. This idea of closing the loop took off around the end of last year, and it has been a game changer in enabling coding agents to work longer productively without human intervention. For example, over the weekend, I was building an app for my daughter to practice typing, and my coding agent could easily work for around an hour, using a web browser to check what it had built multiple times before getting back to me, without needing my intervention. The engineering loop executes quickly. Every few minutes, the coding agent might build and test a new version of the software. I hear frequently from developers who are finding new ways to engineer more effective engineering loops. This is an active area of invention! Developer feedback loop: In this loop, a developer examines the current product and steers the coding agent to improve it. Last year, a lot of developers (including me) were acting as the QA (quality assurance) function for our coding agents, manually finding bugs and then asking the agent to fix them. But with coding agents much more able to test their own code, the amount of time we need to spend on this function has decreased significantly. This allows us to make higher-level product decisions, such as what key features to offer, where the UI needs improvement, and so on. The developer-feedback loop operates over time intervals between tens of minutes and hours — that's how frequently a developer might review a product and give feedback. In the case of the typing app, I changed my mind a few times about the visual design, what cat costumes she can unlock as she learns (she loves cats), and the user flow for a grown-up to log in and steer the child's learning experience. When a developer has a clear vision for what to build, it is still a lot of work to translate that vision into a specification for a coding agent to implement. Further, after the developer has seen an implementation, they might update (or perhaps clarify) the spec to steer it toward what they want. If you find that the system repeatedly runs into certain problems, building a set of evals for the agent becomes useful. AI-native teams are increasingly using AI to help shape product direction, for example, automating the gathering and analysis of usage data, summarizing written and verbal customer feedback, or carrying out competitive analysis. However, for pretty much all the products I’m involved in, I see humans as having a significant context advantage over current AI systems — we know a lot more than the AI system about the users and the context the product has to operate in — and thus humans play a critical role. Many people describe this human contribution as “taste,” but I prefer to think of it as humans having a context advantage, since that gives us a clearer path to helping AI systems get better. This also speaks to why this step can’t be automated: So long as the human knows something the AI does not, human-in-the-loop is needed to to inject that knowledge into the system. External feedback loop: This includes a wide range of tactics like asking a few friends for feedback, launching to alpha testers, or putting the code into production with A/B testing. These tactics are usually slow, rarely taking less than hours and sometimes taking days or even weeks. This data informs the developer vision, which in turn continues to drive the detailed product spec, which in turn drives the coding agent. With coding agents speeding up software development, more engineers are starting to play a partial product management role. For many engineers who are growing into this role, the hardest part is shaping the product vision and striking a balance between building (bridging the gap between vision and spec) and getting user feedback to evolve the vision. It is important to do both! I will write more about how to do this in future posts, but for now, I find it encouraging that engineers are playing an expanded role (just as product managers and designers now do more engineering). [Original text: The Batch]
414
1,643
189
8,538
691,201
Chris Ellis retweeted
I haven't written code in over a year, but I've realized I still find great joy in code smooshing. I find some old code that needs a good smoosh, and I work with an agent to do the deed, leaving it on average 30% shorter and more readable, in my old legacy code base. Reading the newly-smooshed code gives me incredibly good vibes, like back when I used to hand-smoosh it for fun and pleasure. Smooshing code, like with fancy syntax or new libraries or whatever, or just plain refactoring it to be better, is one of the most satisfying things you can do as a coder. And I've realized you can still do it. You kick off a loop and a fleet to smoosh like 95% of your code base, sure. But you reserve the absolute bubble-pop worst old code for your own personal smooshfest. Can't recommend enough.
17
10
3
311
26,249
Chris Ellis retweeted
A big difference between engineers and product people is how they answer if the system "can do" something. Product people answer from a specific situation. Engineers answer from an objective capability. The former is more app-like. The latter more platform-like. We need both.
6
3
2
80
23,388
Chris Ellis retweeted
Almost all AI model and agent progress is downstream from evals. Open weights post training for specific domains comes down to evals. Agent improvements in the applied AI layer is all about evals. Agentic enterprise deployments that actually can augment work is all about evals. It’s all evals. This will become a core competency of any enterprise in the future. The companies that are able to best understand their own (and/or customers) workflows and how well agents participate in that work will be in the best position to actually drive real automation.
54
43
11
464
117,730
Chris Ellis retweeted
Next Week Is CRITICAL For Closed Source AI - GPT 5.6 has to be a massive leap in performance - Fable has to be un-banned Otherwise there will be a massive exodus to GLM 5.2 and Kimi 2.7 starting Monday morning
81
19
3
340
26,122
Chris Ellis retweeted
Pretty remarkable what’s happening with open weights AI right now. We’re seeing models achieve SOTA results on specific tasks, and getting close to frontier on some areas of coding and other domains. The more that open weights is able to maintain only a marginal gap from the frontier, instead of a widening gap, the more value that can be created with AI. Incidentally, this is actually fine for the frontier labs as well; if we can lower the cost of an overall task then AI usage goes up in general. You’re still likely using frontier models for planning, orchestration, reviewing, and other parts of work. But this is all very good for the applied layer of AI, which is now in a great position to cost optimize workloads with cheaper models or use tailored open models post-trained for specific tasks to improve performance.
61
57
7
611
153,823
my todo list has just shifted to Claude Code sessions
2
4
96
Chris Ellis retweeted
Brand design is one of those things where mediocre is almost worse than bad
24
31
5
429
37,411
Somebody release a model
🤖 Made with AI
39
13
2
490
15,482
Chris Ellis retweeted
After Anthropic rolled out the Agent SDK, lots of devs built cool integrations on top of it, because it “legally” used the more generous Claude Code Max subs. Then Anthropic did a rug pull, moving it to API pricing, no warning. And now they’re walking back… for how long tho?
wow - this is huge! anthropic is officially walking back their decision about banning programmatic use of claude code subscription quota why is this a big deal? this is a signal that anthropic is revisiting their ecosystem strategy which many of us have been criticizing by allowing invoking claude code programmatically, anthropic will basically extend their subsidized subscription to power a much wider range of applications, not just their own, which effectively means they are leaning more into being an infrastructure provider rather than the super app that eats everything else they still have more to do to gain back my trust as a developer but this is a very positive change and i'm happy to see anthropic revisiting their strategy
41
10
3
317
61,981
Chris Ellis retweeted
just finished rerunning FC Diamond on my historical charts. none of the official tables/charts are capturing the degree of takeoff. xitter.cf/karpathy/status/206440… its this same chart all the way down difficulty classes (below) breaks every curve fit because Fable is a diffferent CLASS of model, with beeeeeg model smell.
This is a super exciting release - Claude Fable 5 is the same underlying model as Mythos but with added safeguards. The benchmarks are great and it's SOTA on everything by a margin but I'll add that *qualitatively* also, this is a major-version-bump-deserving step change forward (imo of the same order as Claude 4.5 was in November), peaking especially for long problem-solving sessions on very difficult problems. You can give it a lot more ambitious tasks than what you're used to, the model "gets it" and it will just go, and it's never felt this tempting to stop looking at the code at all (but don't do this in prod!). The model still has quirks that people will run into and the safeguards are configured to be a little too trigger happy for launch, which can hopefully be tuned over time. I feel a lot of things changing as working software increasingly comes out on a tap. The Jevon's paradox kicks in and I feel my own demand for software growing substantially. You can ask for anything - explainers, visualizers, dashboards, bespoke single-use apps (e.g. a full wandb that is hyper-specific just for your project), you can 10X your test suite, auto-optimize code, run giant research projects with custom HTML for the results, anything! "Free your mind" (Matrix ref). Really looking forward to all the things people build!
3
10
5
98
36,632
Chris Ellis retweeted
Coding is basically the pinnacle of what you could reasonably automate with AI, and yet we still need human engineers to oversee agents for them to be effective. The AI models are trained on an incredible amount of sophisticated code. The users are highly technical and can use the latest tools quickly. The work is “verifiable” because you can test an app. The outcomes are often removed from the quality of the code (you can have sloppy code but the app can still work). And the context for the agent is often already digitized and sitting in the codebase. That’s an incredible amount of benefits that AI coding agents get to work with. Some of those apply to knowledge work, but most don’t in areas where the work needs to be fully reviewed to be useful, or where data isn’t as abundantly digitized. This makes the job for agents in knowledge work more complicated. So if with all of that, engineers still remain in very high demand, the risks are going to be less than what’s perceived for other areas of knowledge work. Agents will let people do far more than they did before, but the people don’t go away.
I like having a job. So consider this take to be drenched in cope. But as of right now, I think that: coding being a relatively “easy” thing for AI to learn + the existence of many currently employed coders, implies that we’re a long way off from mass while collar disruption.
91
71
16
565
146,263
Chris Ellis retweeted
Writing ADR's for agents has been such a good decision Capturing all the non-obvious decisions in a codebase makes every agent in your stack that touches the stack smarter It's the thinnest layer of docs that captures the stuff code can't
53
47
9
839
59,823
Chris Ellis retweeted
Excited to share our most powerful new Claude Code feature: dynamic workflows! Mention "workflow" in a prompt and Claude will dynamically create an orchestration plan that it strictly follows, allowing you to confidently trust that every stage happens in the right order even across 100s of agents.
350
818
334
7,776
1,705,596