Michael Berg retweeted
Andrej Karpathy:
“A 1B model can be much more effective than most current LLMs.
90% of current models waste a ton of capacity and money on remembering stuff that doesn’t matter.”
here is his 5 core thesis about Small LLM’s:
1. the intelligence was never the knowledge. Strip the memorized internet, keep the reasoning. What’s left is the cognitive core, and it’s small.
2. the core might fit in ~1B params. Maybe less. “Plenty of room at the bottom.” We’ve been paying for a library when we needed a brain.
3. size stopped being the bottleneck. What makes an agent work is the tools it can call and how tight the feedback loop is, not parameter count.
4. intelligence lives in the system, not the weights. A raw LLM is a CPU with no OS. Small core + good harness > giant model alone.
5. you can own it. A model on your hardware is one no one can rate-limit, price-hike, or deprecate on you.
in 12 months the gap between teams shipping on-device and teams paying per-token frontier bills won’t be capability.
TwIL an 8.8B model at 90%+ of frontier isn’t a benchmark flex.
It’s the thesis Karpathy’s been repeating all year, made real.
study more on this below ↓
Modest intelligence is arriving faster than most people realize.
Today, we're open-sourcing TwIL-8B.
Just 8.8 billion parameters, yet approaching frontier-model scores on several difficult benchmarks:
• 93% of frontier performance on coding
• 92% on instruction following
• 90% on contest mathematics
All from a model small enough to run on your own hardware.
But the benchmarks aren't the most important part of this announcement.
For years, the AI industry has focused on building the most capable model, regardless of the cost required to run it.
I believe that equation is changing.
The next era of AI will be defined by the cost of completing work. Not simply the intelligence of the model, but how efficiently that intelligence can accomplish the task.
If a model with 8 billion parameters can reliably complete the same work as one with hundreds of billions, why pay for the larger model?
As smaller models become more capable, intelligence becomes something we can distribute, specialize, and run across the hardware we already own.
That's the future we're building at @thewebAI
TwIL-8B is available today. Open weights.
And this is just the first of several announcements we're making today.
HF Link: huggingface.co/webAI-Officia…
Michael Berg retweeted
People should always be in control of the future.
OpenAI is against the notion that it would be acceptable for people to be relegated to a role of doing things that “AI systems think makes sense to reward in humans”.
People should not cede control of the future to superintelligence, and that requirement should ultimately constrain AI progress.
Michael Berg retweeted
Sama is very challenging for people to engage with because you have to listen to all the words he says in the order that he says them
sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
Michael Berg retweeted
AI was supposed to make us work less
I've never been this tired in my life
most days I have up to 10 agents running at the same time. one fixing a bug, one writing pages, one refactoring something I already forgot I asked for, 3 other I have no idea what they are doing but i suppose i ask it?
and I jump from conversation to conversation all day
compared to 5 years ago, I probably context switch 100x more
by the end of the day my brain is fried. completely toasted. nothing left, not even for the people I love
I call it the fried brain effect
thought it was just me but almost every founder I talk to lately says the exact same thing
we got superpowers, and somehow the pressure only went up. if you can ship 10x more, you feel like you have to ship 10x more. every single day
I don't have a solution
honestly I have no idea where this goes from here
anyone else feeling this?
Michael Berg retweeted
The entire city of San Francisco is spooked and on edge due to AI safety shit, it’s insane how much it’s bleeding into my everyday life. Like I was at my bus stop and I notice a guy looking really stressed out so I said hey, are you worried about misaligned AI too? He took out an AirPod and said “what?” So I said you know stuff like agent swarms, AI escaping RL sandboxes, existential risk. He paused his Barstool podcast and said “sorry, what? What are you talking about?” Just the mention of this stuff made him too anxious to speak, I think. So I said I know man, what?! is my reaction to all this too, and then asked him what his p(doom) is. He then said “dude is this some gay thing or something because I have no fucking clue what you’re asking me” then he got on the bus. People are not handling all this news well. It’s really affecting us. All of us.
Michael Berg retweeted
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Michael Berg retweeted
OPENAI 🔥: GPT-6-Astra model slug has been spotted on the APIs.
If we will actually get it tomorrow, it would be a huge week.
Routing first 👀
Michael Berg retweeted
GOOGLE 🔥: Gemini 3.8 Flash started appearing on Google Coud Console quotas page, a usual release predecessor.
Earlier today, users also spotted that Gemini 3.8 Flash has been powering some of there conversations on Gemini already.
Very soon 👀
Michael Berg retweeted
I think people are overcomplicating agent harnesses.
I've been building backend systems for almost 20 years. When I look at an AI agent harness, I recognize 70% of it.
In simple terms, an AI harness is the infrastructure around the model, and it is what makes it reliable enough for production.
The way I see it:
→ Tool routing looks like an API gateway.
→ Memory builds on storage, retrieval, and context management.
→ Agent workflows are state machines.
→ Checkpoints are durable workflow state.
→ Retries, timeouts, and fallbacks are resilience patterns.
→ Guardrails and tool permissions are authorization.
→ Tracing is observability.
→ Evals test non-deterministic behavior.
→ Human approval is a workflow gate.
We have used many of these concepts for decades.
Now, the remaining 30% is where AI changes the rules.
The model is probabilistic, so the same input can produce a different decision, tool call, or execution path. It can also produce something that looks correct while being completely wrong.
Traditional software executes the path we define. An agent may help choose the path while it is running.
And that unpredictability is why evaluations, observability, permissions, and constraints matter so much.
I think that many of these "ground-breaking" concepts are just old backend patterns with a new name.
If you take a 10,000-foot view, things will start to look similar.
Thoughts?
Michael Berg retweeted
It is hard to communicate how much programming has changed due to AI in the last 2 months: not gradually and over time in the "progress as usual" way, but specifically this last December. There are a number of asterisks but imo coding agents basically didn’t work before December and basically work since - the models have significantly higher quality, long-term coherence and tenacity and they can power through large and long tasks, well past enough that it is extremely disruptive to the default programming workflow.
Just to give an example, over the weekend I was building a local video analysis dashboard for the cameras of my home so I wrote: “Here is the local IP and username/password of my DGX Spark. Log in, set up ssh keys, set up vLLM, download and bench Qwen3-VL, set up a server endpoint to inference videos, a basic web ui dashboard, test everything, set it up with systemd, record memory notes for yourself and write up a markdown report for me”. The agent went off for ~30 minutes, ran into multiple issues, researched solutions online, resolved them one by one, wrote the code, tested it, debugged it, set up the services, and came back with the report and it was just done. I didn’t touch anything. All of this could easily have been a weekend project just 3 months ago but today it’s something you kick off and forget about for 30 minutes.
As a result, programming is becoming unrecognizable. You’re not typing computer code into an editor like the way things were since computers were invented, that era is over. You're spinning up AI agents, giving them tasks *in English* and managing and reviewing their work in parallel. The biggest prize is in figuring out how you can keep ascending the layers of abstraction to set up long-running orchestrator Claws with all of the right tools, memory and instructions that productively manage multiple parallel Code instances for you. The leverage achievable via top tier "agentic engineering" feels very high right now.
It’s not perfect, it needs high-level direction, judgement, taste, oversight, iteration and hints and ideas. It works a lot better in some scenarios than others (e.g. especially for tasks that are well-specified and where you can verify/test functionality). The key is to build intuition to decompose the task just right to hand off the parts that work and help out around the edges. But imo, this is nowhere near "business as usual" time in software.
Michael Berg retweeted
Well this is not right @madebygoogle. So I travel and the phone loses features?
