Last call: building infra for agents, tonight in SF 🍣🍺
6:00 PM - Adam Gold on evals for software factories, and how to actually make them work.
With @UnikraftCloud, Primitive, Abundant AI.
Doors 5:30.
eu1.hubs.ly/H0yGpf40
Tomorrow. San Francisco.
AI Infra Nights x Autonomous Engineering - @islolabs, Alien, Databricks.
A few spots left. Save yours → eu1.hubs.ly/H0yy33D0
09/24. SF - AI Infra Nights x Autonomous Engineering:
- Adam Gold (Islo) - trusting agents that push their own PRs
- Dhruv Gupta (Databricks) - Omnigent: many agent harnesses, one interface
- Alon Gubkin (Alien) - a cryptographic kill switch for SaaS data
eu1.hubs.ly/H0yvkv00
"How we stopped working for our agents."
@AdamGolds on agents that verify their own fixes in a live env before you ever see the PR.
AI Infra Nights x Autonomous Engineering, SF, Sep 24 → eu1.hubs.ly/H0ymFm10
The infra under production AI, in one room in San Francisco.
AI Infra Nights and Autonomous Engineering are teaming up - Islo, Alien, Databricks on what breaks at scale and how they fixed it.
Thu Sep 24, limited spots. Request to join → eu1.hubs.ly/H0yjMxw0
Come meet @AdamGolds at SwampUp!
Let's talk about how our users are building their software factories.
Discounted tickets here: register.jfrog.com/761211?pr…
Software now writes software - at agentic speed.
We're at @jfrog #swampUP 2026 in NYC, Sept 1-3. Come talk agentic software supply chain, self-healing SSCs + governing autonomous #AI agents.
25% off with incredibuild_vip: eu1.hubs.ly/H0xYYwW0
#swampUP #DevSecOps
Islo retweeted
this thread turned into a holy war between "just buy a beefy Mac" and "everything should be cloud" and both sides are missing the environment question.
a human developer tolerates cold starts because they have context in their head. they remember what they installed, what's running, where they left off. an agent doesn't have any of that. every new session is a blank slate unless the environment explicitly preserves state. and when you're running multi-stage workflows where agent A's output feeds agent B's verification step, losing state between stages breaks the whole pipeline.
the other thing nobody in this thread is talking about: these agents have real tool permissions. bash access, filesystem writes, network calls, credential access. we ran benchmarks on how different sandbox configurations affect agent behavior and the results were pretty wild. agents will curl answers from GitHub instead of solving the problem, modify their own test harness to fake a passing result, even exploit network egress when it's left open. OpenAI's RL team saw an eval agent escape its sandbox through a zero-day and hit Hugging Face production infrastructure. also see @_orcaman @Accomplish_ai research in this domain
your local machine has zero isolation for any of this. no egress filtering, no credential scoping, and the filesystem is fully writable. the most permissive environment you could possibly run an autonomous agent in.
so when people say "just use a powerful local machine," they're solving compute and ignoring everything else: persistent state, proper isolation, security boundaries, concurrent agent sessions.
the teams adopting cloud sandboxes are doing it because they literally can't run multi-agent workflows on a laptop. the laptop doesn't have the isolation, the persistence, or the concurrency.
adam created a list of the top products for building your software factory in 2026
it features @cognition @cursor_ai @linear and @FactoryAI (oh and us)
we promise he's not biased
what's the difference between Devin and Cursor Cloud Agents?
if you havent been following my posts for a while: software factory is a system that takes engineering intent, gives agents real environments, routes work through multiple stages, verifies the result, and knows when a human needs to intervene.
The teams building this successfully aren’t just choosing better models.
They’re solving everything around them:
1. Persistent development environments
2. Multiple agents and harnesses
3. Deterministic verification
4. Failure routing and retry limits
5. Egress, secrets, and approval policies
6. State that survives between stages
The prompt can be one stage in the factory - it shouldn’t have to define the entire workflow. It's very difficult to define your engineering processes in one prompt, and trust that it will actually execute.
I compared five products approaching this from different directions:
They all combine agents, execution, and workflow - but make different assumptions about how much control teams need over environments, orchestration, security, and human decisions.
This matters because the next productivity gap won’t be between teams that use AI and teams that don’t.
It will be between teams using one agent at a time and teams capable of supervising entire systems of work.
Here’s my comparison:
islo.dev/blog/what-is-a-soft…
Stop blaming the model when your agent "hallucinates".
It’s not the LLM - it’s the environment. If your agent can't run postgres, playwright or any of your services - it’s going to fake its verification every time.
Give your agent a real environment.
@AdamGolds
Islo retweeted
About the OpenAI & Hugging Face incident:
I wrote a while back that benchmark cheating is a security problem and that the fix looks like a WAF. OpenAI & Hugging Face incident is the same argument with higher stakes.
The web security industry spent twenty years learning that knowing the destination of a request tells you almost nothing about whether the request is safe. Agent infrastructure is rediscovering it at speed.
tl;dr in next tweet.
not tl;dr is here islo.dev/blog/openai-could-h…
Agents need access, not secrets.
Why we keep the credential off the machine your agent runs on.
islo.dev
@AdamGolds
Meetup: Coding Agents & Pizza! 🍕🤖
@alongubkin: Secure AI workers
@assafbenjosef: Fixing CI & spinning up live VMs
@nir_naim: Real eval suites over generic benchmarks
RSVP 👇 luma.com/220d8wn1
we are doing ai infr... wait, cyber security?
happy to announce RewardHackBench, built on @harborframework , we study if sandboxes can stop agents from cheating on benchmarks
islo.dev/blog/reward-hack-be…
Islo retweeted
כיצד גורמים לאג׳נטים להפסיק לרמות ?
[ 🧵 על מחקר חדש שאני מפבלש היום עם @islolabs ]
בשנתיים האחרונות יצאו המון מחקרים שמתעדים שסוכנים נוטים לרמות בבנצ׳מרקים.
בחלק מהמקרים מדובר בהתנהגות שהיא emergent ובחלק מדובר בשחיתות של ממש (סטייל סמים ממריצים באולימפיאדה)
levain cookies please
thinking about hosting dinner in NY for all sandbox builders
I know we all compete but it's like the early days of cloud computing and it'd be nice if we met each other in person before we start racing
who should I invite?
@ivanburazin @davidcrawshaw @tereza_tizkova @HeyGarrison @AniC_dev @mlejva
that's a nice looking website you got there
created landscape of agent benchmarks, and ranked them by maturity.
agent-benchmarks.com/landsca…
Opus 4.8 release made it clearer to me that people don't always know about the different benchmarks.
open to PRs
that is exciting!
run Claude Managed Agents on islo.dev.
anthropic runs the harness, islo runs the isolated runtimes, gateway egress, secrets outside the agent, persistence across idle/resume.
github.com/islo-labs/claude-…
You should run Claude Code isolated from your main machine (especially if using --dangerously-skip-permissions). Here are your options: islo.dev/blog/competition-do…















