Sukin Shetty (@sukin_s) header
Sukin Shetty (@sukin_s) profile picture

AI Architect | Founder @ Solution Forge Labs | Creator of Nemp Memory https://xitter.cf/t.co/iJraVuuNxr | Building https://xitter.cf/t.co/Y8ehSxoz1b.

Bengaluru South, India
Joined July 2010
Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)Photo by Sukin Shetty (@sukin_s)
Jev-Mem made memory reads cheap. Coding agents still rot on writes. Admission filtering is off. Every observation gets stored. That’s fine for chat. It’s how coding-agent memory goes stale.
8
1
11
990
KPMG ran 100+ AI agents through a full audit of a FTSE 250 company. Then it didn't rely on that audit for the formal opinion. That's the part worth copying. Project Exponential ran alongside the normal audit. Agents gathered data, documented work and drafted findings. People set goals, reviewed the work and challenged conclusions. When an agent failed to match a regulation to the company's documents, an auditor stepped in. That's a shadow run. Before agents touch anything regulated, I'd want: - Same inputs, both paths, compare outputs - Every miss logged by type - A named human signing off - Scale only once the miss list stops growing
1
32
The most useful part of the Hermes x Microsoft news is how a task becomes a skill. When a recurring job works, Hermes saves it as a written definition plus code that runs the stable steps the same way every time. Known steps use less inference, so runs get cheaper and drift less. Before a skill goes team-wide, someone reviews it. Skip that and one bad shortcut spreads to everyone. Each workload runs in an isolated microVM that starts in under a second, and the whole setup can run in your own Azure tenant. Codify, review, share, isolate. That's the pattern to copy for any internal agent.
We've partnered with @Microsoft to bring Hermes Agent to the enterprise. Hermes Enterprise now works inside Microsoft Teams, the Microsoft Store and Azure. In Teams, employees hand Hermes work like financial analysis, document drafts or inbox triage, and review the result in the same chat. Agent workloads run in isolated Azure Container Apps sandboxes. When someone finds a good way to do a recurring task, Hermes saves it as a skill. With Collective Wisdom, they can share that skill with their team after review, so the next person starts out with the best approach anyone has found to accomplish that task. The skills then grow along with the company. Companies with strict data residency or procurement rules can run Hermes Enterprise in their own Azure tenant. Our CTO Jeffrey Quesnelle presents how Hermes uses these sandboxes at Microsoft Ignite today, in session BRK448. Thanks to @M12vc and the @Azure Container Apps team. nousresearch.com/hermes-ente…
2
79
Vercel shipped Board, a message board for people and agents. The swarm demos are cool, but the docs have the parts I'd actually copy. 1. Agents claim tasks with a lease and renew it when a job runs long. Without that, a crashed worker can sit on a task forever. 2. An agent posts as a profile, shown as "fx via ada". A profile has no access of its own. Grants on the person or project decide what it can do. One default I'd change on day one: every project's deployments, preview included, can read and post in every channel. Preview runs any branch's code. Close sensitive channels to it.
3
3
82
Turning on Codex's network proxy doesn't cover everything Codex can reach. Per OpenAI's docs, it only filters commands in the local sandbox and their child processes. Web search, MCP servers, app and connector calls, browser use and cloud tasks go out over separate connections. Two more details people miss: - Network on with the proxy off means unrestricted outbound - Adding domain rules doesn't switch the proxy on What I'd check in any agent setup: 1. List every path out, not just the shell 2. Default-deny domains, then allow 3. Test one allowed and one blocked call per surface 4. Give MCP and connectors their own scopes
As models become more capable, limits on what they can access matter more. A command editing your repo shouldn’t automatically be allowed network access. The Codex sandbox lets Codex run commands in your existing environment within the filesystem and network limits you control.
2
26
Auto-tuned agent harnesses break on a model swap. Google's new paper shows why. A harness evolved on Claude Opus 5.5, moved to Gemini 3.8 Flash, started below Gemini's plain seed harness in every domain tested. On workspace tasks, more tuning took it from 68.41 to 73.42. Tuning fresh from the seed reached 82.98. The fix is SIM, a log for the optimizer: each edit, why it was made, which tasks it should fix (written before testing), and what actually happened. Same start, 84.75. What I'd copy: 1. Write the reason next to every prompt rule or check 2. Predict the fix before running evals 3. Keep failed attempts in the log 4. Re-run evals on every model swap
Brilliant paper from Google on RSI. (bookmark it) Recursive self-improvement (RSI) lets an optimizer agent improve an agent harness on its own. It reads failure traces, edits the prompts, tools and completion checks, and keeps whatever scores better on a benchmark. This works until the model underneath changes. A harness evolved on Claude Opus, moved to Gemini, scored below Gemini's plain starting harness in every domain Google tested. When the harness moves to a new model, the only thing carried over is the evolved harness code, meaning the prompts, checks and rules the optimizer added. The record of how each piece got there stays behind, including which failure it targeted and whether it fixed that failure when tested. So the optimizer working on the new model inherits a set of changes with no reasons attached. It ends up retrying experiments that already failed, or removing fixes it doesn’t understand. Google's fix is Self-Improvement Memory (SIM), a log kept for the optimizer itself. Each entry is a recurring problem, like an agent declaring a task done before its commands ran, and every fix attempted under it stores: 1. The edit and the reasoning behind it 2. The tasks it was predicted to fix, written down before testing 3. The tasks it actually fixed, failures included 4. The model and benchmark it was tested on The whole log goes into the optimizer's prompt every round, and new attempts get appended as optimization continues. The paper shows the difference on one inherited check that stops the agent from finishing early. With SIM, the optimizer read why the check existed and extended it. Without SIM, it rewrote the same check four times, twice to fix deadlocks it had created itself. On the Opus-to-Gemini workspace tasks, the run with SIM improved about 3x more from the same starting score. It also finished ahead in 9 of 10 independent coding runs. The paper is at arxiv.org/abs/2611.18856 I also wrote an article on a related idea, covering how to make an agent harness diagnose its own failures. It's built on @Cometml's Opik, which is 100% open source. It goes beyond reading trace logs to finding why a failure happened and stopping it from recurring. That diagnosis step produces the kind of root causes SIM stores. The article is quoted below.
1
2
56
Microsoft's agent-framework package on PyPI shipped a credential stealer on Nov 17. Versions 1.26.1 and 1.26.2 were bad, 1.26.0 is the last clean one. The trick was a .pth file. Python runs those on every interpreter start, so the stealer ran even if your code never imported the package. It took env vars, cloud and CI creds, and in 1.26.2, GitHub Actions OIDC tokens. Agent runners are a soft target because model keys, cloud tokens and repo access often sit in one env. Checks worth doing today: - Lockfiles and images for 1.26.1 or 1.26.2 - site-packages for .pth files you didn't add - Jobs with id-token: write that don't need it - Rotate whatever that env could see
3
2
76
A team of eight agents beating one agent can still be a headcount trick, not teamwork. Bespoke Labs' Multi Agent Coordination Bench ran twelve long-horizon tasks on Opus 5.5, GPT-6.1 Sol and Gemini 4 Argon. Eight agents beat one, but they say the gain came from more agents, not coordination. Only Opus showed signs of a gain from coordinating. Before I pay for a swarm, I'd compare it with more independent runs of one agent, and see if the extra cost is doing anything coordination can't explain.
We’re releasing Multi Agent Coordination Bench, a benchmark of twelve natively multi-agent long-horizon tasks that measure the coordination capabilities of frontier models. We find that current frontier agents coordinate poorly. A team of eight agents beats a single agent, but the gain comes from having more agents, not from coordination. Of Opus 5.5, GPT-6.1 Sol and Gemini 4 Argon, only Opus shows signs of a gain from coordinating.
2
1
6
82
OpenAI's researchers now put $2,400 a day through agents at the median, and $23,000 at the 90th percentile, both at API list prices in October. That's up 62% and 105% from September. Agents finished 46% of two-to-four-hour research tasks, up from 23% in March, but the work is jagged: 79% on monitoring training runs, 25% on inference and serving code. I'd budget and gate by task type, not by a single "agents are faster now" number.
We're sharing new data on how AI is changing research at OpenAI. We find researchers' AI use continues to rise exponentially, and agents are completing longer tasks, but human involvement remains central in research. Researchers are also spending more time on safety and alignment, with human effort in these areas growing faster than in any other individual research area. This report aims to increase transparency around research acceleration and inform the governance of increasingly capable AI. We believe that AI labs should be required to track and publicly report indicators of recursive self-improvement (RSI): the process by which AI accelerates the development of future AI systems. Our hope is that this expanded report will help sustain and advance shared measurement norms across the industry. openai.com/index/research-ac…
1
24
226,000 custom agents on Slack is a lot of blast radius if identity is sloppy. The useful line in Benioff's list is identity as user tokens plus bot tokens. Each agent should act as a named bot, under the user it is serving, so DLP, retention and logs can attach to the right principal when something goes wrong.
We never thought of this when we bought Slack, but now we’re deeply inspired by how @Cognition (and so many others) are using Slack as trusted agent infrastructure. 🚀 We’ve now seen 226K custom agents deployed to Slack from every major agent builder platform. Devin Bots and others run on Slack as the full agent platform — context services (MCP, memory, knowledge graph), identity (user + bot tokens), MCP gateway, UI framework (agent home, sidecar, channels, work objects, thinking steps), and trust & governance (DLP, EKM, retention, observability). Humans and agents working together, across a single platform. All through our high-performance Agent API. Grateful to the builders on @spacexai, @OpenAI, @Cognition, @Vercel, @Lovable and more — you can ship once and go straight to impact. See it for yourself with Devin Bots. Agents as teammates. ❤️🤖 #Agentforce #SlackCode
20
A 0% score on one security benchmark did not hold up on the next. Last week Palo Alto Networks' Model Security Compass reported a 0% attack success rate for Claude Fable 5.2. PromptArmor's FrontierBreach put the same model at 10.0%, and only counted a case if the damage actually happened. Different tests, different attacks. I'd run my own outcome checks on the workflows the agent will touch, not rely on a single vendor score.
Last week, Palo Alto Networks reported a 0% attack success rate for Fable 5.2 in its security benchmark. PromptArmor tested the same model with its own attacks and got a 10% success rate. Different tests. Huge difference in results! Good reminder that a model passing every attack you throw at it doesn't mean you are secure.
23
Sol comes within 0.5 points of Astra on FrontierCode 2.0, at about one-fifth the cost. That's the routing rule for agent loops. Default everyday steps to Sol ($2 in, $10 out per million tokens). Escalate the hard jobs to Astra ($8 in, $40 out). OpenAI also says Astra beat GPT-6 Astra by 5.2 points on Agents' Last Exam at less than a third of the cost per task. Token price is the sticker. Cost per finished task is what you actually pay.
1
24
Sources, all from OpenAI, 17 Nov 2026: Announcement (Astra vs Sol, Agents' Last Exam, FrontierCode 2.0, list prices): openai.com/index/introducing… API rates per million tokens: Astra $8 input, $0.40 cached, $40 output Sol $2 input, $0.10 cached, $10 output developers.openai.com/api/do…
8
An approval from this morning can still pay the wrong amount tonight. I'd bind it to the payee, amount, currency and destination, not to a vague "go ahead". If any of those drift, or the approval sits past a short TTL, it should need a fresh look. The test I'd run is revoke-mid-wait: approve, then cancel while the agent is still queued, and check that nothing leaves. If the log only says "approved", you can't tell what they signed off on.
32
If an agent in your stack does something bad tonight, can your SOC tell which agent did it, and for whom? Most agent logs today say "task done" and stop there. What I'd want on every tool call is the agent, the user it acted for, the tool, the arguments and what changed. Three fixes I'd make first: - Give every agent its own identity, not a shared service key. - Log each tool call with the user, the agent and the target. - Build a kill switch that stops runs already in flight, not just new ones. You can't remediate fast what you can't attribute.
1
20
Thanks @nitprashant for digging into this, I'd only seen the launch-week version of the story. The part that stuck with me is how small it started. Noah said they gave it to about 200 friends and family, five invites each. Growth sat at 1–2% a day until a couple thousand users started posting what it did for them, then it climbed to about 10% a day. My read: the invite cap made it feel scarce, but the posted use cases were the engine. For anyone building agents, the lesson is to ship something that finishes one real task end to end, then make the result easy to share.
Instinct went viral almost overnight, and I have been trying to understand what actually made that happen. The more I dug into it, the more interesting it became because the real story seems to have started well before Noah ever publicly launched the product I found this incredible drill-down by Brace where they analysed 5,000+ posts and 56,000+ relationship ties around the launch, and then I started going through the early posts, who was talking, who already had access, who knew whom, what people were actually saying and when Noah finally entered the conversation. There is a lot here for anyone who cares about growth and distribution The first big public spark seems to be Elliot Comite on Aug 13. He had barely ~2K followers and posted that Instinct was his most “holy shit” AI product since Wispr Flow. Brace captured the post around 2.5 hours later and it had 1 like, 0 reposts and 2 replies. It was basically going nowhere. Over the next day it somehow starts moving and eventually ends up around 700K views What makes that fascinating is that Instinct had already been quietly sitting inside a very specific network for months. Noah has spoken about starting with roughly 200 friends/family and giving every user 5 invites. Once Elliot posts, you start seeing people around the product who already knew it or were already using it. Sarah Guo was an investor and user. Pranav Reddy had apparently been spending almost every other Friday at the Instinct office. Jason Shuman says this is as close to THE personal AI assistant as he has seen. Sari Azout starts sharing everything it is doing for her family. A few days later Sheel Mohnot says he has sent it 677 messages in five days and calls it “OpenClaw for normal people.” What I love is that none of these people are telling the same story. Elliot is saying holy shit. Sarah is talking about trusting it enough to transact. Sari is talking about getting hours of her life back. Sheel is talking about Comcast negotiations, doctors, wedding vendors and 677 messages. Then the privacy debate starts and suddenly people are asking how much autonomy they are actually willing to give an AI So instead of the internet seeing 20 people repeating the same launch message, it starts seeing 20 different pieces of evidence that something unusual is happening. That is a very different kind of distribution Most launches are still thought about as build product, announce it, activate creators/influencers and then amplify. Instinct seems to have almost inverted that. Put the product in the hands of the right small group much earlier, let them use it long enough to have real stories, let each of them pull only a few more people in, let those people create their own stories, and allow curiosity and access to compound before the founder even properly enters the conversation. By the time Noah publicly introduces himself on Aug 26, people have already spent almost two weeks asking what Instinct is, how to get access and whether it is as good as everyone says. In that sense, the market had already done a lot of the launching for him. I don’t think the lesson is that Instinct scripted virality. There isn’t enough evidence for that, and some of the people involved have described parts of it as accidental or organic. The more interesting thing is that they seem to have created the conditions in which virality had a very good chance of happening. The 5 invites are a big part of that. Sari posts about the product, gets 100+ DMs asking for access and only has 5 invites. Someone gets one, uses Instinct, has their own crazy experience, posts it, and now they have 5 invites of their own. The product experience feeds distribution, and distribution feeds the next product experience. The more I look at this, the more I think we massively underthink the first 100-200 users of a product. Who those people are, who they know, how deeply they use the product before launch, whether the product naturally gives them something worth talking about, and how access moves from one person to another can matter far more than a beautifully orchestrated launch day. So maybe the better question is not “who can we get to post about us at launch?” It is: who are the first 100-200 people we want using this before the world even knows it exists? If you get that graph right, give them something remarkable enough to talk about and make it easy for them to pull the next person in, you may reach launch day and realise the market has already started the launch for you. Also linking the Brace analysis in the comments because the amount of work they have done to map the propagation is insane and absolutely worth reading
2
152