reward hacking @primeintellect
sf | nyc
Joined February 2015
- Tweets17.7K
- Following1.5K
- Followers53.6K
- Likes107K
will brown retweeted
Prime agent #2 on FrontierMemory ranking after Letta Dream!
Great to see Prime Agent do so well on this new self-improving memory benchmark.
My only nit: Let's add agent versions to results!
FrontierMemory also allows measuring the ability of agent harnesses to improve over time. We find that Letta Dream and @PrimeIntellect's Prime Agent improve the most from learning from prior experience data.
will brown retweeted
we are hiring applied researchers to work on scientific AI at prime intellect. you'll operate at the frontier across scientific domains with some of the most ambitious companies and labs building in the space - join us!
applied research at prime intellect is both a client-facing team and a fundamental research team. continual learning is our north star. we view this dual approach as necessary in order to study the core challenges faced by models which self-improve in production. the real world is big and messy.
we spend part of our our time working with customers to train SOTA models for specialized domains, and we automate and productize our work as we go. we also do open-ended research on novel methods for self-improvement and long-horizon memory, and we get to experiment with data that no other lab gets to see.
today, the most readily-deployable form of continual learning is converting "exhaust" (agent traces, logs, feedback) into robust environments and tasks for continual RL. but this is not the final form. there are major open challenges around knowledge acquisition, harness-weight tradeoffs, user modeling, dynamic underspecified preferences, updating and planning across timescales, selective forgetting, data efficiency, learning from golden answers, and company-scale simulation. this is the roadmap, roughly speaking.
some of these may require changes to architecture, or parameter-efficient methods, or fundamentally new algorithms. we have our hypotheses and early evidence, but everything is on the table. the world is our oyster.
we are betting on a radically different future of what ai deployment will look like, and we can see the path more clearly every day. it is a world where agents learn like humans do, with billions of individual models compounding their learnings from the real world continuously.
we're hiring incredibly ambitious researchers in SF, NY, and London. come join us.
will brown retweeted
@willcb is deriving me from first principles tomorrow..
We are going to need an L0.5 for continual learning
will brown retweeted
applied research at prime intellect is both a client-facing team and a fundamental research team. continual learning is our north star. we view this dual approach as necessary in order to study the core challenges faced by models which self-improve in production. the real world is big and messy.
we spend part of our our time working with customers to train SOTA models for specialized domains, and we automate and productize our work as we go. we also do open-ended research on novel methods for self-improvement and long-horizon memory, and we get to experiment with data that no other lab gets to see.
today, the most readily-deployable form of continual learning is converting "exhaust" (agent traces, logs, feedback) into robust environments and tasks for continual RL. but this is not the final form. there are major open challenges around knowledge acquisition, harness-weight tradeoffs, user modeling, dynamic underspecified preferences, updating and planning across timescales, selective forgetting, data efficiency, learning from golden answers, and company-scale simulation. this is the roadmap, roughly speaking.
some of these may require changes to architecture, or parameter-efficient methods, or fundamentally new algorithms. we have our hypotheses and early evidence, but everything is on the table. the world is our oyster.
we are betting on a radically different future of what ai deployment will look like, and we can see the path more clearly every day. it is a world where agents learn like humans do, with billions of individual models compounding their learnings from the real world continuously.
we're hiring incredibly ambitious researchers in SF, NY, and London. come join us.
will brown retweeted
Excited to announce that I've joined @PrimeIntellect!
Our mission is to build the open training stack and let people own their intelligence. Prime Intellect has always gone a step beyond open-weight models and strived for a fully open-source ecosystem by sharing everything you can think of: training infra, data, evaluations, inference, algorithms, harnesses, and more.
In just a few weeks, I’ve already learned so much from an incredibly goated team with deep expertise across the entire model stack.
Looking forward to building frontier open models with the team. Watch out for what we are cooking, big things ahead!
will brown retweeted
Today we're releasing prime-rl v0.10.0 and verifiers v0.4.0
We added our new RL algorithm IPO as the default, tailored for async RL stability, as well as performance improvements and new model support
will brown retweeted
we just spun up a full k8s cluster inside our sandboxes thanks to the new LAN feature: private networking between a cluster of sandboxes.
leaving it running over the weekend to prototype an idea
back Monday to check the hillclimb results.
agent swarms will replace traditional software development. enterprises will order hyper-optimized systems end-to-end, unattended.
will brown retweeted
More thoughts
There is an endless number of lists of people who are good at devrel devx growth engineering etc but what they are all missing is WHY. so I will give you a few of my personal opinions
1. @charles_irl at modal the perfect example of someone who is an engineer first but deeply cares about education. Before modal, Charles was an educator and it’s extremely apparent in how he approaches his content.
2. @trq212 at Anthropic is actually really humble about his growth with the fact he works on Claude Code and that people are naturally excited about Claude code. But one analogy that Thariq shared that I think rings very true for why he does good work is this idea of reaping and sowing. You must reap (do a lot of research, experimentation, talk with customers, hands on engineering) before you can sow (create good content). There are too many people who skip the reaping and go straight to sowing slop.
3. @philipkiely from Baseten is phenomenal because it’s one thing to create content but it’s another to build the systems that turn that content into high value pipeline generation. Two many people miss this second step of thing your work to baseline revenue and overly rely on “vibes” of doing good work. Anyone would be lucky to have this TOFU machine.
4. @mattyp at spaceX is extremely consistent but has also mastered the home setup/ camera investment that he’s able to produce good quality content very efficiently and fast in order to stay with the times.
5. @ad0rnai at prime intellect is very good at making videos! But her special power that more people need is an attention to detail. To many it will feel like overthinking when she hosts and event or films a video but she will play at your hidden heart strings in ways you don’t even know to make you fall in love with her work.
I have a lot of thoughts on the whole “it’s impossible to find someone technical who’s also good at creating content, has taste, and deeply understands the ai ecosystem (twitter)”
In no particular order
1. I have seen many companies hire for this role too early. When I interviewed for this vague new media/ devrel/ DevX world before joining Cerebras, many very cool companies reached out but they had no clear product to market and no clear audience.
What ends up happening is that there is no clear need or scope and this person is dumped to do everything because they lowkey can do everything. (Be warned)
2. Many of the best folks at this role did not start as GTM folks. The most common profile are folks who are engineers first, generally have CS degrees (although many are now self taught) and naturally found themselves really excited to be educators as well. Key word = educators.
I had a CS degree from MIT, decided very early I did not want to be a software engineer because it was too monkey mill, and carved out a hybrid role for myself.
3. When thinking about who is exceptional at this role, it’s important to distinguish the person from the company. For example, if someone is at a top lab, how much of the viewership, followers, etc are from them or just natural company groupies. (This is not to say folks at oai, ant, etc are not extremely talented).
4. So! Now everyone thinks they can now be a creator because how good AI is. One of the main differentiators for engaging content is having a literal human face in your content versus obscene amounts of screen recordings and graphic animations. Thus everyone is trying to hire a face of a company and needs to hire faces of a company.
5. This is just a thought but there are very clearly AI companies built around their founder where the founder is the face and everyone at the company feels like it’s their job to just make the founder more famous. I will not name names. I appreciate smaller companies like @browserbase who seem to amplify the many voices in the company.
6. There are still a ridiculous amount of companies who do not realize the value of investing heavily in these types of folks. Perhaps they are scarred by the lack of impact of enterprise/ old school marketing. But these AI companies, when not doing well, react by firing and downsizing these technical GTM teams (wrong move buddy), you never hear about them again, and they spiral into oblivion. Don’t do this.
7. There are two types of companies who “get it”. The companies who just do, and the companies who have been burned by 5+ years of failing to get the company message out and are finally like oh shit. We need to hire these types of folks and do this type of thing.
will brown retweeted
Had a great time collaborating with @NVIDIAAI on Nemotron 3.5 Super VL!
AutomationBench is a great testbed for evaluating agents on complex, real-world enterprise workflows. So much more to build here, excited to keep pushing! 🚀
will brown retweeted
Together with NVIDIA we post-trained Nemotron 3.5 Super VL on thousands of tasks
Reflecting enterprise workflows in sales, operations, marketing, finance
We improved the AutomationBench-AA score by ~70% on the public split
will brown retweeted
Open superintelligence is built in the open
Kickoff party for Open Source AI Week with @huggingface
will brown retweeted
Introducing filesystem checkpoints for Prime Sandboxes:
Checkpoint a sandbox at any point in a trajectory and restore it into as many fresh sandboxes as you need.
Fork new rollouts from any intermediate state, then checkpoint inside a branch and fork again to dig deeper.
will brown retweeted
idk they’re consistently releasing the best models known to humankind. they might be on to something
will brown retweeted
Second release from the RL Residency today: NVFP4 on non-Blackwell Nvidia GPUs.
Very cool work from @secemp9! More coming soon.
for ~5 months in the @PrimeIntellect RL residency I've been working on HINT4, a (hopefully) better NVFP4 quant method that runs natively on both older and newer cards. none of it would've been possible without their generous compute and advice secemp.blog/2026/10/13/hardw…
will brown retweeted
Moondream is literally a two man frontier lab. Super excited to bring @vikhyatk & @jason_allen onboard to scale multi-modality and inference at prime!
will brown retweeted
I've seen how powerful auto-research can be. To make models better at it, we need environments to train them in. And what better subject than mechanistic interpretability?
@amit05prakash and @infinitasium have created just that, and written a beautiful blog post about it.
Can AI agents do interpretability research?
At @PrimeIntellect Residency program, we built BioCircuit, an RL environment where agents trace attribution graphs on Qwen3-4B, find the features behind a concept and get graded by intervening on prompts they've never seen.
Link : amitprakash.in/biocircuit/
