i don’t understand a lot of things. but i want to.
Joined November 2023
- Tweets1K
- Following339
- Followers3
- Likes5.5K
omelette catching up retweeted
PhD Advising vs Vibe Coding #3
(Note: these posts are about how the way I work is changing. I find that some of the skills of being a PhD advisor translate well to the way I work today with AI and some don't. Since others are going through this transition, I decided to share my thoughts.)
I can supervise 6 PhD students and feel on top of things, even during deadlines. At 8, I'm still ok but working like crazy. I've gone way above this at various points and then I can't do the job the way I want to.
I would have trouble advising 6 AI projects at once. AI agents work quickly and come back with answers or questions that need my analysis or input multiple times a day.
I guess I could meet each agent once a week like I might do with a PhD student, but that would defeat the efficiencies of using AI. So, with all my other responsibilities, 2-4 projects at a time seems about right.
omelette catching up retweeted
Graphics Programming weekly - Issue 458 - October 11th, 2026 jendrikillner.com/post/graph…
omelette catching up retweeted
Back when calculators were a little more controversial, v/@JonErlichman.
omelette catching up retweeted
Reasoning From Scratch round 8, the finale!
This last lecture in the series covers distillation as an efficient alternative to the previous reinforcement learning training runs.
(Planning to take a brief break, but I will be back!)
00:00 Introduction and series recap
01:07 Why distill a reasoning model?
02:51 Soft vs. hard distillation
11:06 On-policy vs. off-policy distillation
18:09 Preparing data for reasoning distillation
22:35 Loading the DeepSeek R1 MATH dataset
28:04 Inspecting reasoning traces and final answers
31:27 Tokenization and chat templates
35:58 Pre-tokenizing the training dataset
39:01 Sequence lengths and the 2,048-token limit
42:38 Creating training and validation splits
45:32 Loading the Qwen3-0.6B base model
46:24 Computing answer-token log probabilities
54:10 Cross-entropy loss for distillation
57:25 Training and validation loss helpers
1:02:03 Implementing the distillation training loop
1:07:26 Running the standalone training script
1:09:41 Plotting training and validation losses
1:11:31 Evaluating checkpoints on MATH-500
1:14:09 Comparing teachers and distillation results
1:16:34 Future directions for reasoning models
1:19:41 Bonus exercises and next steps
omelette catching up retweeted
You don't even need any money. You just need to check out Joseph Suarez's state space branches and figure out how to make the sokoban solve not take any domain specific priors and then sweep it across everything else
omelette catching up retweeted
Notice the cost column
So I tested DeepSeek V4.1 Flash 1017 on real work. 2 repos. 105 hidden bugs. Find and fix what you can.
The first results:
Muse Wonder Full 1.0 (max): 42
Opus 5.5 (max): 41.7
Gemini 4 Argon (high): 40.7
DeepSeek V4.1 Flash 1017 (max): 34 🟣
Grok 4.7 (xhigh): 29
Qwen3.8-Max (max): 25.7
DeepSeek V4.1 Flash (max): 21.7 🟣
GLM-5.3 (max): 19
A great improvement. Strange that the model wasn't labeled V4.2.
763B params. The weights are already on Hugging Face.
I expect the score to change by ~1.7 points after repeating the test 3 times.
More data and results at different effort levels dropping in this thread very soon 🧵
omelette catching up retweeted
Splatting is going to be a critical part of the ego collection pipeline IMO, so I'm working on improving our current official support.
code here - - <github.com/rerun-io/examples…>
The current shipped version does well for less than 1M splats, but not so much for larger ones such as the iconic garden scene (5.2M splats). I ported over brushes kernels to @rerundotio and now on my M5 I'm getting ~40-50fps when recording or a solid 60fps when not. This is basically a 10x improvement from the current version and this huge improvement that wouldn't have been possible without all the verification loops we've added with the rerun skills and MCP. I'll break down what was required to make this work ⬇︎
I wanted to share something I've been working towards since forever. I've finally built out a full slam + hand reconstruction pipeline that works IN REAL TIME at 30fps on a single rockchip 3588. All streaming directly to @rerundotio from a @BitRobotNetwork robocap. This is similar to what a Quest3 or Project Aria 2 provides, but opensource 🙂
I'm so excited I finally don't have to really on closed source software like the meta quest or others. I finally have a realtime performant pipeline that works on an edge computer, but importantly can also scale to a high performance one when needed.
omelette catching up retweeted
If anyone's interested in what I've been up to lately, I've been leading development of some 3D foundation models at @Roblox—like a next-gen texture generator, released today: devforum.roblox.com/t/beta-i…
4k resolution, PBR materials… and it's free. :-)
Also gets some things right that are important in practice (and annoying with other tools), like generating good texture even in spots where different parts of the mesh intersect.
Was happy this required not just ML, but some classic signal processing to really get right. Great work, by a great team.
Has definitely been a journey of understanding how generative AI tools can be designed as just another level of abstraction for creation—rather than a “one shot” replacement for artists. Still a lot of work to do, and more to come…
omelette catching up retweeted
In July we open-sourced a dynamic graph of everything a child learns in primary school. 1,590 concepts, 3,221 dependencies.
Now we’re turning every concept into interactive experiments kids can play and learn from.
First one, age 7: heat and water.
Everything that used to be a boring textbook can now be an engaging experience, potentially tailored to every child.
Kids education just got a massive upgrade.
omelette catching up retweeted
The fact that female erections are just now entering the conversation at the same time as flying cars being released is also a perfect representation of how far behind we are in female health.
omelette catching up retweeted
🦔Amazon launched two new Bee devices today that record the conversations around you whenever they’re switched on. The $75 Bee Frontier goes on your wrist or clips to your shirt, and the $100 Bee Eclipse snaps onto the back of your phone. A $12 a month plan adds unlimited summaries and ties the whole thing into Amazon’s store.
My Take
I think this solves a problem for Amazon more than for the person wearing it. Amazon raised its 2026 capex guidance to $220 billion and needs that AI spending to start earning money, and a recorder that hears you mention dish soap and then puts it on your homepage is about as direct a sales funnel as it could build. For years Amazon sold Echos at cost just to get a microphone into your house. This time you pay $75 for the device and $12 a month on top of it.
I’m not sure who asked for this. Most of what anyone says in a day isn’t worth writing down, and your phone already makes a to-do list for free. Bee also records everyone in range, so your coworkers and your kid’s friends end up in the log without ever agreeing to it. If those conversations are worth that much to Amazon, it should pay people to wear these instead of charging them $75. I'm just tired of everything listening...
Hedgie🤗
omelette catching up retweeted
Linus Torvalds says, rules prohibiting AI usage are “UTTERLY STUPID” and “idiotic”.
For the Linux kernel, Torvalds says, “we’re not going to make stupid rules that say ‘you can’t use tools for X’”.
omelette catching up retweeted
Jensen Huang on why founders don’t need to be right from day one, but they do need to be able to learn quickly:
“The technology that founded the company turns out to be exactly wrong.”
“I went down to Fry’s, and I bought three textbooks... I brought them back to the company. I gave them to the engineers.”
“Everybody would have thought that NVIDIA started out as world leaders in 3D graphics. We learned it from a textbook.”
“The big lesson for me is technology’s changing all the time. So long as you’re able to confront reality, so long as you are able to learn, the technology itself actually doesn’t matter. Since then, NVIDIA has been inventing all kinds of technology."
"We approach everything with the same attitude. If it’s important to do, we’re going to go learn it. How hard can it be? It always turns out to be much, much harder than we expect. But you go into it with the attitude, how hard can it be?”
omelette catching up retweeted
Muse Spark Open -- weights releasing today -- runs great on new Mac Studios thanks to work with @johnternus and Apple.
