pushing the frontier of model capabilities | ex Eng @mercor_ai, @uwaterloo
San Francisco, CA
Joined July 2014
- Tweets91
- Following415
- Followers456
- Likes300
So bearish on all these new data brokerage companies with zero domain expertise in RL, no lab is going to assume the risk of purchasing your dataset without the providing the training ready environment and calibrated tasks to prove difficulty. The reality is 99% of the value comes from task creation on top of the seed data
Ryan Hu retweeted
One of the first features we built into Terminal-Bench (long before Harbor existed) was --livestream to watch the tmux session.
Glad to see @kobe0938 cooking up the better, more generalized version for Harbor :)
I’m seeing a uptrend in labs in-housing rl environments, but I still remain bullish on independent vendors.
The bottleneck isn’t training. It’s acquiring high-signal data which get turned in environments.
Vendors working across multiple labs benefit from stronger feedback loops than any individual lab can generate internally.
I predict instead of labs closing off to external vendors, RL becomes less vertically integrated than people expect.
A handful of vendors own the engagements with labs, then subcontract work and data downstream.
Join Anthropic and help Claude learn real-world knowledge work! You'll own the strategy for domains (e.g., finance, healthcare, legal): source high-value tasks, design RL env / reward signal, and evaluate the capability improvement.
Look for Staff+ research engineers who love applied research AND getting hands-dirty with datasets. RL / reward design experience required.
Our team also closely partners with product to ship Claude in Office, Cowork, and Claude Design
job-boards.greenhouse.io/ant…
Now that post train bench came out we need:
Data annotator bench
RL Environments bench
Data delivery bench
Researcher bench
Continual learning on steroids
The RL environments space is just a group of new grads and drop outs who vaguely know each other and pull all nighters to deliver data
What a joke, the worst possible decision you can make is work at a no name yc startup for a below average salary and equity. If you’re young, hack on your own idea or work at a unicorn. This is exactly why startups complain of a talent shortage, you get what you pay for
This quoted post is unavailable.
It is insane that AI is not being adopted with urgency in the healthcare and life sci sector. From first hand experience, there are the brightest minds in this vertical working on both the model training and data curation. Take over the incumbents and empower the people
this is actually insane
> be tech guy in australia
> adopt cancer riddled rescue dog, months to live
> not_going_to_give_you_up.mp4
> pay $3,000 to sequence her tumor DNA
> feed it to ChatGPT and AlphaFold
> zero background in biology
> identify mutated proteins, match them to drug targets
> design a custom mRNA cancer vaccine from scratch
> genomics professor is “gobsmacked” that some puppy lover did this on his own
> need ethics approval to administer it
> red tape takes longer than designing the vaccine
> 3 months, finally approved
> drive 10 hours to get rosie her first injection
> tumor halves
> coat gets glossy again
> dog is alive and happy
> professor: “if we can do this for a dog, why aren’t we rolling this out to humans?”
one man with a chatbot, and $3,000 just outperformed the entire pharmaceutical discovery pipeline.
we are going to cure so many diseases.
I dont think people realize how good things are going to get
We use modal sandboxes as the infrastructure for all RL environment projects. Running eval and training rollouts has never been easier. Switching to @modal changed my life fr
Over 1 billion sandboxes have been launched on Modal.
Since launching three years ago, we've seen Modal Sandboxes become foundational to how AI is being built.
Today, teams like @Lovable, @tryramp, @cognition and more are using Modal Sandboxes to power everything from coding platforms and background agents to RL infrastructure at scale.
Long horizon tasks mean multimodal agents. It’s not enough for an agent to code — it needs to retrieve the ticket from linear, figure out context from slack, spin up a local dev environment on the computer and test UI using the browser.
Very insightful takes in this article. Models will never stay static and agent frameworks assume so. Reliable enterprise agents will be RL’ed in simulated production settings with the tools that emit raw control (bash, cdp) so workflows are always recoverable.
Ryan Hu retweeted
Software agents can self-improve via self-play RL
Introducing Self-play SWE-RL (SSR): training a single LLM agent to self-play between bug-injection and bug-repair, grounded in real-world repositories, no human-labeled issues or tests. 🧵
