Is there a real moat for all of these sandbox companies? Feels like everyone is pivoting and the product is always the same??? - Fast boot time - High clock speed - Scale up and to zero But like beyond lightweight microVMs it feels like the technical depth is fairly shallow? Like everyone is running on the same CPUs, and you can't really optimize the actual workloads because its just random agent-generated code at runtime. Aside from race-to-the-bottom on pricing, how do you compete?

So, we're mostly not writing code by hand anymore. But, would you hire a SWE that had never hand written code?

the big discount on intelligence arunabhdastidar.substack.com… “We stopped the individual engineering subscriptions to Claude and Codex on the engineers’ Macs, and routed all engineering work through the API. The team and backlog didn’t change, and we didn’t ask the engineers to work more. Before this, five engineers collectively paid $800 to $1000 per month for Max and Pro plans. After this, the bill was approximately $1000 per day for the same work. So in one fell swoop, the cost went up thirty times.”

Before joining a team or starting a project, it's important to know which tribe of agentic engineering you're choosing: - The software engineers: Treat agents like another software engineering problem. Borrow ideas like Map-reduce to enable parallel agent swarms. Build pipelines, workflows, and focus on modularization. - The ML practitioners: Agents are just another ML system. Regularization techniques work whether it's in weight or token space (e.g. clipped gradients vs skill-opt). Good evals are meant to be hill-climbed on. E2E agents are preferred. - The management experts: Agents behave roughly like a human, so all human organizational principles apply. Consulting terms like MECE (@wulfie_bain_) finally have a chance to breathe in technical work.

We’re about to spend most of October on the road, and we’d like to meet you at one of these stops. Here’s where we’ll be: Oct 17: BSidesNYC, New York Oct 19 to 24: BSidesAugusta week, Augusta, GA Oct 27 to 29: [un]prompted II, San Francisco Oct 29 to 30: LASCON, Austin Augusta is the big one. On Oct 21 and 22, Leo Meyerovich and Sindre Breda are teaching a two-day hands-on class on AI agents for security investigations. It’s $595 and registration closes Oct 14. Then on Oct 24, Leo gives a talk on two years of SOC CTF evals for investigation agents. If you’re a SOC analyst, an engineer building agents, or just someone who likes graphs, come find us. Bring a hard investigation, a strong opinion about AI agents, or both. We learn more from what’s breaking for you than from anything we’d put on a slide If you’ll be at any of these, message us and we’ll make time.

GPT-6 Astra and Claude Fable 5.1 are out, and they're pretty compelling if you have the budget. But put them into a real investigation harness, and things get interesting. On CyBT-CTF, 59 incident-response cases worked in Splunk, Astra reaches 69% and Fable 5.1 reaches 61%. GPT-5.6 Sol, an older and smaller model, reaches 76% when running through Louie. The same GPT-5.6 Sol scores just 58% in Codex. Astra is the strongest coding agent we've measured, but only gets to 69% at high thinking effort, at $2.34 per solved case. Fable 5.1 gets 61% at both high and low effort, at $1.19 per solved case. So far, the choice of agent seems to matter at least as much as the choice of model. We haven't run Astra or Fable 5.1 through Louie yet. That's the test we're most interested in next.

I will say, at Snorkel we have excellent discipline about quality, and I attribute a lot of it to (i) our smart team; (ii) a “look at the data, understand the data” culture!

The latest botsbench results on the release of GLM 5.3 family are newsworthy not in intelligence for agentic AI investigations, but cost. Run on cybt-ctf, our 59 task SIEM investigation benchmark, we see that Z.ai’s effective answer to OpenAI’s Luna is GLM 5.3 Flash. Also, the story on quants makes GLM 5.3 Flash more interesting as well. As a reminder, just a few months ago, Z.ai’s GLM 5.2 release was the first time we measured an open-weight model performed at frontier levels on botsbench.com investigation tasks. Despite that, OpenAI’s 80% price cuts on GPT-5.6 Luna meant we still preferred OpenAI Luna over GLM 5.2 as both smarter and cheaper. With the new GLM 5.3 family, we are not seeing smarter results on botsbench: the important story is that GLM 5.3 Flash pricing means frontier open-weight AI has effectively caught up to GPT-5.6 Luna in quality and cost. GLM 5.3 Flash means surprisingly little hardware goes a long way. It also gives an extended shelf-life to older GPUs. For the first time, we’re reporting a quant on botsbench: our own GLM 5.3 Flash W4AFP8 that makes it more manageable on older GPUs. Out-of-the-box, GLM 5.3 Flash ships as a 643 GB BF16 release, which typically requires 8 x H200s to run inference. We squeezed to 177GB to run on 2 x H200, with little impact on botsbench scores. For those who are curious, we quantized the routed MoE experts to int4, paired with static float8 activations, and left attention, shared experts and the dense layers at block-float8. The int4 experts are 88% of the weights, so compressing those while leaving the precision-sensitive paths alone cut the size to 177 GB. The main reason we picked this approach was to avoid requiring the small floating-point units only available in recent GPUs, as many of the teams we work with that self-host have GPUs that are a few years old. On GPQA-Diamond the quant scores 90.9% against 91.2% published for the full-precision model. On botsbench, it scored 35/59, compared to 36/59 unquantized. Note that we avoided any cyber-specific fine tuning or abliterations, which bring concerns of overfitting and testing for more unexpected loss of function. Our quant shows general open-weight models are running at quality levels cyber teams would have loved from frontier models last year, and require only a couple older GPUs. Also of note, the top botsbench investigation scores are still from combining the Louie harness with proprietary models - and those lead with a sizeable margin. However, Louie with GLM 5.3 Flash breaks into the top 5 of the global leaderboard, and lands noticeably ahead of regular coding agent harnesses with other models: GLM 5.3 Flash is both good and cheap.