20 • Scaling RL envs thru AWMs • It is our moral obligation to make an impact in the world proportionate to our potential
Joined September 2022
- Tweets3.3K
- Following355
- Followers442
- Likes16.2K
laxman retweeted
Introducing opencomputer - the fastest coding agent in slack
once you try it there's no going back - give it a shot on opencomputer.dev/factory
laxman retweeted
super sick! process rewards using a universal verifier (jev) - hopefully it's adversarially robust enough
Excited to release Step-Jev 🔥
step-by-step rewards for agent RL, even when there’s no answer key
After DeepSeek R1, everyone said step rewards get hacked, so just reward the final answer.
We found the opposite. With an AI judge grading only the final answer, our agent learned to fool it and accuracy dropped every single update until it was worse than no training at all.
Step rewards fixed it. Everything is open!
laxman retweeted
Help Step-Jev, my reward is stuck (plateauing) 🥵
Excited to release Step-Jev 🔥
step-by-step rewards for agent RL, even when there’s no answer key
After DeepSeek R1, everyone said step rewards get hacked, so just reward the final answer.
We found the opposite. With an AI judge grading only the final answer, our agent learned to fool it and accuracy dropped every single update until it was worse than no training at all.
Step rewards fixed it. Everything is open!
laxman retweeted
Interesting insights.
1/ why do small models post-train so well? a thread on the "few bits" paradox 🧵
2/ RL reward ≈ 1 bit per rollout. LoRA touches <1% of the weights. the update is tiny by every measure
3/ but behavior changes completely. assistant vs autocomplete, solver vs guesser
4/ the answer: pretraining already learned every persona and strategy. post-training just raises the probability of the good ones
5/ early RL mostly sharpens what's already there (pass@100 → pass@1). sustained RL might build new strategies on top. the debate is open
6/ implication: a well-pretrained small base + your own traces + a cheap step-level judge = frontier on your task
7/ post-training is the new pretraining. agent traces are the new oil 🛢️
laxman retweeted
Starting a series on how you can achieve the building blocks for this software factory on @opencomputerHQ serverless agents
Opencomputer gives you the building blocks to create your own factory with maximum flexibility:
- Memory
- Durable sessions
- Database
- Schedules + triggers
We will create a bunch of agents iteratively and build on top of our template until we have the full picture achieved. The first part of the template is the coder agent, which leverages opencode to perform pull requests.
Next up we will add memory to our coder agent. From there we will see how to include a
verification agent so that the PRs are also verified in a real computer for correctness (visual, functionality)
We will then create the PR review agent which checks the work of the coding agent
Next will be the oncall agent and the performance agent, two agents that check production metrics and react to incedents and suggest improvement performances
Bonus agent: the dreamer agent, which can run nightly, reflect on the memories and clean them up. It also has access to all the other agents' codebases so it can suggest PRs to improve itself and other agents!
Here's what OpenAI's agentic software factory looks like, today.
Details: newsletter.pragmaticengineer…
(thanks to all the OpenAI folks who explained how it works! And Perf Factory looks especially interesting to me)
laxman retweeted
I was waiting for this…
laxman retweeted
1/ Excited to release The ultimate guide to multi-harness RL
The same model behaves differently in every agent harness. So we built an open way to train any model with RL on any task set, inside the harnesses people actually use, like Claude Code, Codex, and OpenCode, without changing a single line of harness or training code.
Trained across four harnesses, LFM2.5-2.6B went from 42% to 54% with 31% fewer tool calls. 🧵
laxman retweeted
automating fine-tuning is happening.
There will always be a tradeoff between generalists and specialists. The tradeoff is a physical reality: look at life's diversity. There will always be room for both. Jev is a wonderful generalist that is almost perfect (maybe too perfect for its own good) at solving the cold-start problem of getting the data to train a specialist. Of course, Jev doesn't permit that use case, but a Jev-like generalist model that does will come out.
laxman retweeted
Interesting work here 👇
laxman retweeted
Laxman is incredibly cracked. You should check this out!
laxman retweeted
Jev + Post-training seems promising.
laxman retweeted
finally theres a better way
laxman retweeted
ngl the part that got me wasn’t even beating jev
we trained it on another verifier’s labels and it basically learnt the judgement itself
feels kinda stupid to keep paying a huge model to make the same decision forever once you have enough traces lol
laxman retweeted
been doing a bunch of post-training at Bandr and needed a reliable way to turn agent runs into training data.
so we built Bandits :)
traces → tasks → outcome evidence → tested verifiers → evals + training data
we're demoing it at Unicorn AI Summit on Sep 21. come find us
laxman retweeted
0→1 is hard.
People will open source you. Thought bois will elegantly pick apart your product choices. Poasters will say nobody will ever use it, but also that it’s a commodity. That you’re the symbol of the top. That you’re cooked cooked cooked.
Until you’re inevitable. Then the idea was obviously good to anyone with eyes.
Fuck em and keep shipping. Coming up with one great idea means you can come up with the next thousand, while the replicators / NPCs are still struggling to copy your first.
And if you’re a mad observer, the best revenge is building something better—not extinguishing the flame of genuine new thinking, product craft, and the beginning of a new cycle for consumer software.
Build mad. Build happy. Just build.
laxman retweeted
your agent is one tool call away from a DELETE you cannot undo.
observability shows you what happened AFTER.
With agents taking actions this is JUST NOT ENOUGH.
Introducing Realtime Steering with Failproof AI, made for AI Workers.
Find failure points, ship policies and enforce them BEFORE your agent can make a mistake