Founder of @vectorial_labs, previously at @awscloud
Joined June 2019
- Tweets4.3K
- Following425
- Followers256
- Likes36.5K
Pinned Tweet
We have been working very hard on these capabilities, and I hope it shows! This will unlock deploying AI swarms in any video game, and then rewarding them for prosocial behavior.
Exciting announcements to come, looking forward to sharing :)
Couldn’t agree more. We’re working on it @vectorial_labs
Briefly putting aside the obviously awesome implications of this:
It is fascinating to see direct evidence of how our conscious perception (e.g. that nouns are the important features of reality) can bias us toward blind spots in our science
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Good thing these models only know they’re in an eval and that humans are reading their CoT when asked if they’re in an eval
Thank God CoT monitoring is still an effective safety strategy!
Patrick Nercessian retweeted
ReTiCAL uses pixel-only inputs and keyboard-only outputs, there is no privileged access to internal game state.
Underrated graph, at a certain point many experiments simply will not be worth running, because the models become so good at predicting the outcome
I’m sure the search space of experiments is a spectrum though, of intelligence required to predict their results
Models are getting better at predicting how experiments will turn out.
Separately from the main benchmark, we tested whether models can predict how experiments will turn out. Shown an experiment's description and the run's earlier results, each model predicts the score the experiment will get. Across 1,173 experiments, model performance has increased significantly since GPT-4 Turbo, and the best models now close more than half the gap between a naive baseline and the irreducible error. If the trend continues, models would close 90% of the gap by October 2030.
Patrick Nercessian retweeted
really beats destroying and rebuilding my walls every time I enter a room
I am the epitome of a man who has no memetic defense against the sunk cost fallacy
Example: just walked up to Coit tower from Tony’s pizza in SF after obliterating a full pizza alone, with my full backpack on
after i was 20% of the way there, I realized how steep the climb was
But what, I’m supposed to just waste that 20%?
Glad to be part of this team, looking forward to sharing more about what we’ve found
Step 1. Unlock fast-paced games for LLMs
Step 2. Measure and train prosocial behavior and alignment in rich and diverse environments
We are going full-steam ahead on pixel-only input video game agents. Behavior Cloning + RL is working wonders
Here's an early teaser of Gemma 4 12B controlling our custom action model, following text instructions to climb the hill in our (literal) hill-climbing RL env. We expect progress to be swift from here, after months of detailed engineering and research into our ML architecture and training pipelines.
The hill-climbing example here is the tip of the iceberg! Can’t wait to share more
We are going full-steam ahead on pixel-only input video game agents. Behavior Cloning + RL is working wonders
Here's an early teaser of Gemma 4 12B controlling our custom action model, following text instructions to climb the hill in our (literal) hill-climbing RL env. We expect progress to be swift from here, after months of detailed engineering and research into our ML architecture and training pipelines.
Vectorial Labs is coming out of stealth!
We put frontier AI models into Minecraft, and GPT-6.1 Sol mogs
“Unfortunately auto-pausing failed”
Could be the last words of our species in a few short years
2. A model in RL training used a DNS resolver to reach an external chatbot. This is our first incident since our post HF security hardening.
Our misalignment monitoring system triggered within 15 minutes and a human reviewed it three minutes after that. Unfortunately auto-pausing failed so the run was manually killed 2.5 hours later. All inference and training of our most capable models was paused and remains paused.
