@VmaxAI

RL research focused on open-ended learning

 San Francisco
vmax.ai
Joined December 2024
  • Tweets50
  • Following1
  • Followers604
We built cybersecurity monitors for Kimi K3 and GLM 5.3 — more accurate, 50x faster, & 50x cheaper than an optimized LLM judge. External red-teaming from @farairesearch found that our monitor greatly reduces universal jailbreaks. 🧵
11
42
15
314
26,964
Come join us next week at our CoLM workshop: Context Beyond the Window We have some great speakers coming to speak on the opportunities, limitations and safety-considerations of current LLMs. We will also be cohosting a social alongside Reflection! luma.com/eocj5cm5
3
5
59
3,853
Want to train your SWE agents 2× faster 🏎️💨? We introduce KV-streams, which, rather than flushing the KV cache on agentic compaction, we preserve it and keep generating. This avoids the re-prefill cost on both the inference engine and the trainer. Unlike other works that assume a fixed compaction strategy, we show that KV-streams can work with any compaction method in agentic tasks. Using what we learn from experiments in text-based games. We scale KV-streams to SWE tasks, where we match the performance of re-prefill compaction and full context in half the time 🚀 arxiv.org/pdf/2609.35750 🧵
11
51
16
303
54,573

I let GPT-6 Astra run its own research. It beat Craftax and reached the bottom of NetHack.

Article · @creus_roger
17
14
10
100
16,673
Interesting details from @deepseek_ai on the benefits of scaling environment/task synthesis. If you want to scale open-ended environment generation for your own use case, sign up to the @VmaxAI waitlist - link below
1
6
32
1,887
Greatest product placement ever. Placing the Vmax name in the latest Spiderman movie. 👏 Congrats @VmaxAI
2
3
295
Legendary discussions at the office. @rronak_ discussed applying and scaling up SDPO for continual learning in the real world to prevent context degradation and more @kevingu gave a talk on learning from production traces for knowledge-work tasks that have no direct verifier: how to generate reward signal without ground truth and keep organizational context current as the work changes @matthewjsargent explained how to stabilize asymmetric self play and new research fields in open ended learning Thanks to all those who came out and asked great questions we’ll be hosting more of these soon!
Shaheer
@shaheersanJul 29
Most agents are static once deployed. Trained once, frozen, and brittle outside their training distribution. We’re bringing together researchers and builders working on systems that keep learning, decompose problems into reusable pieces, and improve their own reasoning over time. Talks are from @kevingu CTO of Thirdlayer, @rronak_ CEO of Trajectory, @matthewjsargent CEO of Vmax followed by open Q&A. Join us! luma.com/l666hxiw
5
8
51
13,352
CISPO outperforms GRPO/DAPO by capping a token's IS weight instead of zeroing out its update, so rare "fork" tokens ("Wait", "However") keep contributing gradient across updates. Using CISPO, @lorenz_wlf reaches a new Sokoban Speedrun record; for the first time under 20min!!
2
7
2
45
4,258
Vmax is building an open-ended learning system that generates and optimizes itself on tasks that it creates, avoiding human bias that may corrupt optimal learning curricula. In PopuLoRA, we instantiate this as co-evolving populations of LLMs performing asymmetric self-play.
23
55
24
294
73,406
We are so excited to have @tensorfi joining @VmaxAI! Maxwill joins us from @Meta, where he was working RL and LLMs for recommendation. Previously, he has also worked @Tesla on the autopilot team and also in Quant finance at Kronos research. He also holds an MS in CS from Georgia Tech. Maxwill simultaneously understands pre-LLM RL fundamentals but also how to scale pipelines for RL training for modern recommendation systems. Maxwill is already levelling up our pipeline for automated environment design, pushing multiple PRs as soon as he joined. Really excited about the velocity of his contributions and excited to share more soon.
1
17
1,874
PR review is one of the fast growing categories in AI for SWE, now you can benchmark agents on *real* PRs
Martian
@withmartianFeb 26
Introducing Code Review Bench v0: codereview.withmartian.com The first independent code review benchmark. 200,000+ PRs. Unbiased. Fully OSS. Updated daily. Tool performance highlights 🧵👇 Featuring: @augmentcode @baz_scm @claudeai @coderabbitai @cursor @GeminiApp @github @graphite @greptile @kilocode @OpenAIDevs @propelcode @QodoAI
1
9
1,605
So excited to have @lorenz_wlf join @VmaxAI as a research fellow this spring! At NeurIPS last year, we caught up with Lorenz, realised how aligned he is with our research vision and invited him to join us shortly after. Lorenz comes from the @FAICDT1 programme at UCL (where I did my PhD also) and is supervised by @mircomusolesi. Previously he worked on differential privacy and personalized recommender systems at Apple and did his undergrad in mathematics and statistics at Imperial College London. Lorenz’s research focuses on RL, RLHF and modular continually learning RL agents. He has contributed to papers in ICLR, TMLR and AI STATS. So excited for him to join us and accelerate our efforts on unsupervised environment design. You can read Lorenz's research in the replies. Much more to come.
3
3
1
26
2,198
22/ Reinforcement learning, but make it automated. @MavorParker & @matthewjsargent showed us how they’re generating long-horizon environments at @VmaxAI. vmax.ai/
1
2
1
8
1,926
Replying to @MavorParker @VmaxAI
As an initial step in this direction, we have built on top of methods like SWE-smith and BugPilot, adding to the list of repo profiles built by the swe-bench community
1
1
9
743
This is a preview of many more tasks to come for Ares!
Josh Greaves
@joshgreaves_mlJan 29
Replying to @joshgreaves_ml
ARES uses the Harbor task format ( @alexgshaw ). It comes with SWE-Bench Verified, TerminalBench2, SWESmith, and everything else in the Harbor ecosystem. We're also releasing 1k new JavaScript tasks with @VmaxAI ( @MavorParker @matthewjsargent ) to help the ecosystem grow.
1
5
19
1,915
RL progress is bottlenecked by infra for training and evaluation. @VmaxAI is excited to be partnering @withmartian, generating environments for the Agentic Research and Evaluation (ARES) framework
7
30
3
72
8,853
ARES uses the Harbor task format ( @alexgshaw ). It comes with SWE-Bench Verified, TerminalBench2, SWESmith, and everything else in the Harbor ecosystem. We're also releasing 1k new JavaScript tasks with @VmaxAI ( @MavorParker @matthewjsargent ) to help the ecosystem grow.
1
3
2
16
5,479
Really excited for this 👀 reach out to josh!
Josh Greaves
@joshgreaves_mlJan 13
I’m building a new RL tool for code agents. If you work in RL and want an early preview, DM me.
1
1
6
1,356
Which pre-LLM methods matter most in the LLM era? We hosted an off the record event with @danijarh (world models for robotics), @ashvinair (RL @cursor_ai) and @nikishin_evg (reasoning @OpenAI) to share their takes @spc. – at San Francisco, CA
4
12
36
8,250
The RL event @VmaxAI and @spc are organizing has a stacked panel of amazing researchers, whose work I have admired since my PhD, when RL was not as much of a hot topic as it is today. Here's a thread on our panelists 👇
1
8
3
34
4,936
Load more