Hopefully you'll get to robotics once tinygrad is savely set up towards all abstractions being "from the book". Fixing RL scaling (no exp. data needed) is THE blocker for robotics. Self-plug: I will publish a lib for fast & simple "Atari from pixels" RL research soon. Stay tuned!

Bought quite some AMD stock based on your investment (thanks btw!). Will maybe add some QCOM. Could be too late because margins are shrinking fast in the OpenAI/Anthropic business model that currently pumps up GPU valuations. Markets do not price that in yet imo

Wild that even with the clearly successful AMD case study you have to push so hard to lead the horse to water. That inertia shows how overstaffed these companies are. Hopefully coding agents will lead to more tiny corps!

Qualcomm acquiring modular maybe shows some better management? If they would approach tinycorp this could be an even funnier story arc than the AMD-driver-umr stuff 😅

Are you bullish on RL finetuning of LLMs going forward? It is sufficient to squeeze out perf (math & code) but catastrophic forgetting bites back. Feels like for image (instead of token) input LLMs are too far from Kolmogorov complexity to be scaled to AGI

Cool! Any idea what causes those drops in throughput?

Makes these nice loss curves even more impressive 💪

Are there 1-2 things from the things you listed that took the most effort?

The Curse of Dimensionality: The Cure of Composability

Can you share specifics about the new OaK implementation? Options and Knowledge is clearly what we currently lack in RL! Instead of learning one blob of weights a composable set of learned things is needed. E.g. what OOM parameter count is it? Thousand, millions or billions?

"Atari Breakout in 4 seconds" and Physical Atari are two extremes. The first is too easy (not using pixels but [ball_x, ball_y...]). The second is too hard (takes too long to iterate on it). Currently working on something that tries to fill the gap...will take a few more weeks

The best part is...i have to wait for 6 hours. The pitfall of RL research in one sentence

The models show respect because they know that they are all slowly converging towards the AIXI model 😄

Does the exabox fit in a bike shed? Would love that picture just for the lolz 😄

And this is the one true reward: ∃

He should reconsider: Selling chips w solid firmware and OSS stack on top -> other people fixing the OSS stack -> possible alternative to CUDA moat CUDA is why nvidia is worth that much

These straighforward comparisons to LLMs are an insult to the brain. The brain runs so much more efficient software. LLM vs brain is like Windows 11 vs TempleOS. I liked the estimate in your blog post "Brain FLOPS": 162 GFLOPS.

Oops, this one should work: github.com/wwu-mmll/deepmrip…