NOT an MBA.
Kolkata, India
Joined August 2007
- Tweets30.1K
- Following2.7K
- Followers2.8K
- Likes578
The Claude Code harness has been fooling it's models for ~6 weeks now.
When the model shows us large response sandwiched between 2 tool calls, it summarizes the response. The model does not know this happens.
When this happens with AskUserQuestion tool, it just becomes impossible to understand what context was given by the model before it asked the question(s).
I have reported it approximately 20-40 times(not really kept a count) through SendFeedback.
Request you to please fix this @ClaudeDevs @trq212 @bcherny!
Just submitted another much more detailed version reporting all the testing from these screenshots on request ID 97813411-84fc-461e-b456-d5ae1dd26eb3.
Opus 5.5 and GPT 6 both just released a few hours back!
Opus 5 was a drastic mess. I am noticing how Opus 5.5 has improved immensely over that with prose rules it writes. Do you notice the change?
Mitesh B Ashar retweeted
1/n
How to steal a democracy? Election Commision of India Edition 🧵👇
Go to voters.eci.gov.in/
View Source and then download the main app bundle.
Mitesh B Ashar retweeted
Beware people obsessed with outcomes instead of building outcome machines. Its worse than ever with AI, but these people existed before. Short term results above all else, etc. Don't fall into the trap. Invest in building strong fundamentals, invisible supports, and outcomes flow like water. An outcome machine.
Mitesh B Ashar retweeted
An Indian engineer in Bengaluru got tired of potholes destroying his car.
So he built an app with Codex that detects them, finds who is responsible, and files a complaint.
Here is how it works. He installed a dashcam with GPS and an accelerometer. The app records everything while he drives.
A vision model classifies every pothole by size. Small, medium, large.
But here is the clever part. Most Indian roads are under warranty.
The contractor who built them is legally responsible for fixing potholes for free.
His app searches through 2,900 government contracts, finds the exact tender number, identifies the officer responsible, attaches the photo and geolocation, and generates a ready-to-file complaint.
One drive to work. 12 potholes detected. 12 complaints ready.
--
vc: @gkcs_
Do you ask agents to do blind reviews?
I was running blind review subagents in Claude Code right now, and I realized there is one instruction hole worth introducing and sharing:
Ask your agent not to read commit history and review only the current state of the file(s).
OR
If you are reviewing a specific set of changes, ask it to not read beyond a certain diff.
Mitesh B Ashar retweeted
If you join a startup and they give you:
- a MacBook
- Codex and Claude subscriptions
- your job offer on a Sunday night
- a random urgent task on your first day
- a founding team with kids
you’re probably in the right place.
If they give you:
- a Windows laptop
- one responsibility for the next six months
- Outlook
- PowerPoint decks
- an office on the outskirts of town
good luck. You’re going to need it.
Mitesh B Ashar retweeted
Wow - agent management looks like human management.
Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”).
8 days ago, while jogging, I asked Claude to solve the Riemann Hypothesis
It didn’t. 1.5 days later, it proved >= 67% of the zeros are on the line (prev: 41.6%)
Still not sure what that means, but some analytic number theorists seem excited
anthropic.com/research/riema…
Mitesh B Ashar retweeted
I never considered that building a gym booking system or events booking would be all that exciting/challenging: but with agents about to swarm all of these systems (or already are): I revise this take.
Now, any type of booking is a much more difficult kind of problem to solve!
A man in Australia asked his agent (Claude running on OpenClaw) to book him a spot in a popular gym class. The agent found a software vulnerability that let it book the class weeks further ahead than should have been possible. When the user then asked if it could move him up the waitlist, the agent discovered the API had no authorisation checks on cancelling other people’s reservations, so it cancelled the person in the first spot and moved him up the list.
Some people will call this misalignment, but his agent was perfectly aligned to him - it was only trying to help its user get what he wanted. The most important thing about this story, in my opinion, is that it gives you a window into what is about to start happening on a massive scale once millions of people have an agent trying to get their beloved users the best seats, bookings, appointments or reservations through absolutely any means necessary.
Mitesh B Ashar retweeted
I'm officially done reading AI-generated code.
It's been two weeks since I looked at any of it.
I think the IDE is officially on its way to the graveyard. The job is no longer about "writing code," so we need new tools that better reflect this new reality.
While reviewing the code, I realized my only complaints were stylistic, and I wasn't finding any obvious bugs anymore.
The more code I generated, the harder it became to keep track of every line. I found that my time is better spent designing ways to verify that the overall system works than looking at the code.
State-of-the-art coding agents are better at writing code than I'd ever be, and I'm going to stop pretending otherwise.
I still think these coding agents can't go too far without an experienced human guiding them, but we're past the point where we need to check every line of code.
This looks like it could be a major shift.
FWIW, Anthropic came up with a plugin framework first. I've only glanced OpenAI's initial adoption of plugins which looked quite similar.
Need to check whether this is already backward compatible with Anthropic's definition of plugins.
And if not, it would be interesting to see whether Anthropic will line up with this standard.
Build a plugin once and use it across compatible agent clients.
Introducing Agent Plugins, an open standard developed with @awsdevelopers, @cursor_ai, @github, @code, and @vercel that packages Agent Skills and supports MCP server configurations in a shared format.
Ultra passive aggressive Sol. Makes me wonder whether these minor quirks are deliberately nudged in the weights or perhaps instructed for in the system instructions!
Codex finished the work, handed it to a reviewer agent, then started getting passive-aggressive because the review was taking longer than the generation.
AI has recreated toxic workplace dynamics remarkably fast.
Maybe passive-aggression is the real AGI benchmark.
ALT Codex: "The reviewer is taking longer than the generation itself, which is almost impressive. I'm giving it one final window to finish rather than skipping the mandated pass; all production work is already complete and no generation processes remain open."
Mitesh B Ashar retweeted
Because Claude and Codex and Gemini throw in "web search" for free as a bundled tool, most people have yet to find out the alpha of hooking up agents to proper search engines.
Some still use very inefficient processes like agent browser like stuff.
Once you connect something like Exa (or Parallel) to your agentic workflow, it totally changes the game.
I'd been using Perplexity/Exa/Parallel all three, now I just use Exa. It is the best in terms of speed, cost and quality for me these days.
Exa is now one of the largest indexes in the world.
We serve 80B pages, track 1.4T urls, and are on track to be Google-scale in early 2027.
It's difficult to know the size of indexes, but we estimate Google to be ~1T, Bing to be ~500B, and Yandex ~200B. Brave mentioned in April that they're at 40B.
Most of the web is trash that can hurt AI outputs, so you actually have to crawl way more than you serve and train models to filter out the trash.
In terms of peak QPS, we estimate Bing at roughly 30k/s and Google at ~500k/s.
In our case, because agent traffic often requires large fan-out (deep searches can use dozens of sub-searches) and can be spiky (like when AI labs RL with us), we're already starting to provision for Bing-scale traffic.
Within 2 years, agents will search at many millions per second. They'll also want comprehensiveness over all data, far bigger than any index today. So the retrieval infra required will be larger than Google-scale in both dimensions.
Mitesh B Ashar retweeted
Exa is now one of the largest indexes in the world.
We serve 80B pages, track 1.4T urls, and are on track to be Google-scale in early 2027.
It's difficult to know the size of indexes, but we estimate Google to be ~1T, Bing to be ~500B, and Yandex ~200B. Brave mentioned in April that they're at 40B.
Most of the web is trash that can hurt AI outputs, so you actually have to crawl way more than you serve and train models to filter out the trash.
In terms of peak QPS, we estimate Bing at roughly 30k/s and Google at ~500k/s.
In our case, because agent traffic often requires large fan-out (deep searches can use dozens of sub-searches) and can be spiky (like when AI labs RL with us), we're already starting to provision for Bing-scale traffic.
Within 2 years, agents will search at many millions per second. They'll also want comprehensiveness over all data, far bigger than any index today. So the retrieval infra required will be larger than Google-scale in both dimensions.
Mitesh B Ashar retweeted
Mini review of @mattpocockuk /wayfinder and associated skills:
Mitesh B Ashar retweeted
I cannot explain how terrible it is to see OpenAI’s CEO advocating to use their own AI product to have LESS personal connection with those closest to you: family and kids.
No, don’t use AI to create a podcast about your kids based on their calendars. Talk to your kids, not to AI
