gpu middle-class | nyc, once upon a time @uwaterloo
nyc
Joined July 2015
- Tweets3.1K
- Following1.5K
- Followers4.3K
- Likes7.8K
Pinned Tweet
I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier.
Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack.
Next items on my checklist: adding more complex environment examples, supporting multi-gpu + async RL, and QoL fixes.
I might scrap external runtime dependencies (Huggingface PEFT + vLLM) and write purpose-built, simpler versions from scratch if I feel the need.
If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.
omkaar retweeted
Tinker changed how people do post-training. We're big fans, and so are many of the teams we work with. But as those teams scale up, they keep raising the same two problems: per-token pricing gets expensive quickly, and you’re limited to the models and features the provider decides to support.
We think a good API deserves more than one implementation.
Today we're releasing Spindle, an open-source, Tinker-compatible training server that runs in your Modal workspace. Your existing Tinker scripts run unchanged, and because you own the server, you pay for GPU time instead of tokens.
omkaar retweeted
I'm looking for a founding engineer [ SWE ] for Ambition.
If you're an avid tinkerer, love learning new things and extending your capabilities, I'd love to talk to you. Pay and equity is top of market with a lot of upside. Looking for people with deep technical curiosity and a history of ramping up on new domains and going deep.
Needless to say, your Ambition card will be issued on day 1.
omkaar retweeted
Kimi K3 is live on Modal.
Moonshot has shipped the world's first open 3T-class model, and we're a day zero launch partner.
We trained a custom DFlash speculator for K3's novel architecture so you can run it faster, losslessly. The most capable open model we've worked with by far.
omkaar retweeted
Should we be worried about how good AI is getting at coding autonomous drones?
Introducing Drone-Bench, a benchmark where AI agents code drones to complete a simple autonomous surveillance task.
Drone-Bench is independent but based on Project Pilot, our work with Anthropic.
omkaar retweeted
Inkling by @thinkymachines is now available on Modal, backed by a custom DFlash speculator for 67% higher throughput and interactivity.
Running on Modal Auto Endpoints with SGLang today.
Today, we are introducing Inkling.
Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.
thinkingmachines.ai/news/int…
Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵
all the AI replies focused on degraded battery and energy usage, maybe all same model?
People replace their phones every ~4 yrs. This means there are hundreds of millions of old phones discarded each year that are still perfectly usable as computing devices. @Google in collabration with @UCSD is exploring how to turn these old phones into cloud-computing “phone clusters”. Putting phones back in service in this way can directly reduce the environmental footprint of computing by avoiding the need for further raw material extraction, and taking advantage of the embodied carbon already incurred from manufacturing these devices, and modern phones actually are already quite powerful computers. Read more in the blog below ⬇️
Grateful to be a small part of this! Rishi and team were so fun to work with
in case this helps someone, my claude.md was simple. something along the lines of:
"""
{brief summary of eval task}
{what is being measured and ideally what the agent does to get 1.0 reward}
In a loop, run the eval on our harness, sift through the logs of our new run in a subagent and try to find any evidence of reward hacking. once you do, reason about what change to the environment can fix this. Patch it and run the eval again.
"""
Long-horizon evals are tedious to get right, often the biggest blocker is how long the eval runs itself. meta loops are nice. Sandbox companies are going to make so much money because evals will only run longer
