Christian Vestergaard retweeted
very interesting, they pruned 2/3 of the experts during mid training then did logit distillation from a larger inkling model
i would expect this to be a big part of why they get so much intelligence for the limited number of parameters
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index.
thinkingmachines.ai/news/ink…
it's crazy going to your parents house and hear them talk about AI as a fun toy that can change their profile pic on Facebook. the world is not ready for what's coming!
2
71
if you take ZONOS3-Mini from @ZyphraAI and give it expressive sample voices from a paid API like Elevenlabs, then you have a great open weights TTS model
1
2
139
Christian Vestergaard retweeted
what i’ve come to understand and hold true in my head but am always revisiting…
1. we don’t have enough raw resources (memory, cpu, power, data centers) - this won’t go away for perhaps up to 6 to 10 years
2. There is minimal to no moat for the prop labs; except selling poison and cure. They will make money selling expensive expensive cyber tokens in response to the security problems created by AI
3. AI fundamentally has changed work and there’s no going back. The box is open and businesses need fewer humans to run a businesses.
4. AI is an economic munition. If a country depends upon AI (and they will) and their access is suspended then a country will be economically crippled by such sanctions.
5. Openweight models are typically three months behind prop labs. They are a critical counter to $ant.
6. Countries must develop their own sovereign open weight models which codify their culture/customs to protect/encode their nations value structures to protect against cultural homogenization.
7. Distillation is competition and fair game. How it is done is should be the focus of conversation (not “if” it’s allowed) - ie distillation currently happens through theft/credit card fraud - that isn’t good!
Though tough thoughts am pondering about:
8. Training on pre-existing IP is perhaps not theft of IP. The same way that a human reading a book, seeing an idea, learning concepts and then remixing that concept isn’t a copyright violation. Mathematics isn’t IP/copyright theft.
this is cool
We believe every company should run its own custom models. But right now that means paying for a GPU by the hour.
Today we’re launching Cerebellum Labs (YC W27). Custom models, priced per token.
$25 in free credits to start: cerebellumlabs.ai
2
155
Christian Vestergaard retweeted
this is it. 1000% it.
people who think something is too late are usually not founders. one could argue that book business was done and amazon appeared to change the game. blockbuster was the king when netflix was the cd delivering peasant til the empire was overthrown.
again and again when the non founders think the game is over, the founders appear and change the game.
it's an infinity war if you choose to believe it, and the game is always on.
Scott Wu on the nihilism creeping into tech: people saying it's "too late" to build an independent company because the labs will just do everything.
His take: those people aren't founders. Founders are rationally optimistic with zero evidence they'll succeed.
If you think coding agents look like this forever, you're wrong. There are 10 more generations of this product to build.
there's so much moat in shipping with extreme intention now that LLMs make things abundant. I'm a big opponent to the focus on shipping more PRs and quicker. they are not bad per-se (!), but the metrics tend to attract people with little intentionality, I think.
4
120
Christian Vestergaard retweeted
The evals are not ok
New NeurIPS paper: Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking.
Evals drive deployment, safety standards, and research. Yet we ignore noise from prompt, judge, agent harness etc, & so underestimate error in benchmarking & annotation, & it gets worse with more data.
I'll get to what this means for CIs & how to fix, but first let's talk about how this makes benchmark hacking easier 🧵
It's mind boggling to imagine how we got any work done before LLMs. I'm taking it for granted.
1
3
148
On-policy distillation should be renamed office hours. Everyone would immediately understand the concept.
Christian Vestergaard retweeted
To make something *truly* great, you always have to put in more time and effort than you want.
The good news is most don’t, so if you do, it’s all the more likely it’ll stand out.
It's time for new training regimes in speech. We believe audio-native RL environments with fine-grained rubrics (naturalness x task completion) is part of the answer. We've been doing research on this and can soon release a preview of our own post-training. Truly exciting times!
Christian Vestergaard retweeted
We believe most interaction with AI will happen through speech. But models still talk like machines.
This is a data problem.
Today we're launching @agenticdataco (YC F26): real-world speech data + audio-native RL environments so speech models sound and act human.














