words fail me @ScuderiaFerrari

AI might make being a generalist useful again... if u can learn fast enough

appreciate you enabling my procrastination @duolingo 😭

some Astra benchmarks barely improve at higher thinking levels... which sent my PCN-obsessed brain down a rabbit hole hypothesis, some curves, and where I might be wrong teendifferent.substack.com/p…

idea → product time has decreased a LOT with Astra… im actually pretty impressed with it from my usage so far with proper guidance its super reliable…and now i kinda get why its one-shotting a lot of things im building something rn and with Sol / other models the first versions were usually kinda subpar…they could build it, but i had to keep iterating a lot before it actually felt complete with Astra the initial version basically just…worked end to end 😭 most of what im changing now is personal taste + small refinements rather than fixing fundamental stuff especially UI + handling multiple states…this is a huge one imo state management is usually what makes an app feel like an MVP vs something actually production-ish…loading states, empty states, transitions, errors, different flows etc previous models could handle some of it but once things got complex you kinda had to spoon feed them / iteratively fix things Astra seems much better at keeping the whole thing coherent switched to the $100 plan…been doing some pretty heavy usage on High with ~270k context and honestly that feels enough for me 75% still left + got 2 more banked resets 😛 feels guud 😀 haven’t tested it much for research yet…got some GPU kernel + model architecture ideas i wanna experiment with next for stuff that isn’t app building tho…Medium already feels like the perfect daily driver

ok im done opening twitter 😭 somehow we went from “openai is experimenting with recursive depth” to “openai abandoned safety” in like 6 hours… this app is just personally rage baiting me at this point

yeah exactly… i’m genuinely confused why reusing the layers suddenly becomes a safety event 😭

saw this random scene from Columbus and it sent me into a weird rabbit hole lol started with attention vs interest... somehow ended up reading about curiosity, productive failure, cognitive debt, creativity, and whether da vinci with chatgpt would be 10x better or just... different wrote it down here teendifferent.substack.com/p…

was messing around yesterday evening with a dumb question... how much do trained weights actually care about the exact activation function sitting between them? did a tiny activation surgery experiment on Gemma and wrote down what happened + some things it made me wonder about teendifferent.substack.com/p…

one thing i misunderstood about cerebras initially...i thought it was basically tenstorrent taken to wafer scale there is definitely shared DNA...local SRAM, explicit data movement, 2D mesh / NoC style thinking, trying to keep compute fed instead of leaning on a giant transparent memory hierarchy but the programming model is meaningfully different with tenstorrent you’re still explicitly orchestrating a pipeline...reader/data mover kernels pull tiles through the NoC, circular buffers synchronize producer/consumer stages, compute kernels chew through 32x32 tiles, writers move results onward cerebras pushes more of that orchestration into the dataflow graph itself...PEs are placed physically, communication uses colors/routes, wavelets arrive on those channels, and that arrival can activate a task bound to the channel so a lot of “what happens next?” is encoded in the topology + bindings ahead of time rather than repeatedly reconstructed by software at runtime rough mental model: CUDA → launch N threads Tenstorrent → move N tiles through programmable stages Cerebras → define where data flows and what becomes runnable when it gets there not saying one is universally better...Tenstorrent’s programmability is probably a feature for many workloads, while Cerebras gets really interesting when the computation graph is regular enough that you can aggressively predeclare communication but then i wonder how long these assumptions survive if the model itself gets weird... we talk a lot about test-time scaling now...more reasoning tokens, longer context, external memory, agents writing their own little markdown brain etc...useful obviously, but there is some limit to how far “just give the same frozen model more context” can take you remembering something in context is not the same as actually learning it what gets interesting to me is a model that can evolve quickly while it runs...bounded working memory + external memory sure, but also some kind of fast internal plasticity...changing state, routing, maybe even local weight/gradient updates from its own experience then inference stops looking like this nice mostly-static graph we spent years optimizing around...maybe the next model isn't just doing a different linear transformation...maybe it is changing the linear transformation while you are talking to it and then suddenly the hardware question gets fun again...how much of NVIDIA / Tenstorrent / Cerebras survives a model whose computation graph, memory and even parameters can evolve online? no idea what that architecture looks like yet...but kinda beautiful that at the bottom of all this insanity we're still asking what a bunch of linear transformations can become

it's funny...when tiny open model beats something 5–10x larger ...“benchmark maxxed” sometimes probably true...but there’s a nerdier possibility people underweight we might still be pretty bad at filling a fixed parameter budget with capability Allen-Zhu/Li’s knowledge-capacity experiments get toward ~2 bits of factual knowledge / parameter at saturation...but how much you actually recover moves with training duration, architecture, signal/noise etc then Densing Law basically asks the same thing from the capability side...effective model size / actual model size...and that density keeps climbing so capability / parameter clearly isn’t some fixed constant when a tiny one suddenly starts bullying much larger models, benchmark contamination isn’t the only hypothesis...maybe the weights are still kinda hollow...not literally sparse or empty,just nowhere near optimally packed and frontier models make this even more interesting...Sol is already around/above Fable on several evals while being much cheaper...rumor is Sol is smaller too, although parameter counts are private so that part is still lore i’m more curious how far we can push C/N before increasing N at all

random thing I keep noticing with kimi k3… it’s weirdly consistent about breaking long shell commands into proper multiline args even inside tool calls... most models just kinda dump the whole thing in one line lol might be coming from how aggressively they cleaned/formatted code data before training, or something reinforced in post training such a tiny thing but now I can’t unsee it

funny timing…we recently started moving our Actions jobs away from @github hosted runners to @useblacksmith 😭 mostly because their runners have been ~2x faster for us, with way faster and more consistent job pickup times but outages like this are a reminder that we’re still dependent on the GitHub Actions control plane lol @useblacksmith any plans to support other CI systems too? 👀