reverse engineering intelligent (learning) systems. rl, evals, safety @emulated_ai
San Francisco, CA
Joined February 2019
- Tweets170
- Following159
- Followers422
- Likes745
Hey Claude, build a league of legends style game and then pit Astra against itself in a 5v5 using computer use.
Ryan Peters retweeted
total gnomic victory
Meet Mistral Large 4, aka Le Chonk.
• 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
• State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding.
• Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure.
• Available to all via API today. Working with cybersecurity partners privately.
Open weights release end of October.
Ryan Peters retweeted
Our clinical trial participants have now used their Neuralink implants for over 50,000 hours, building one of the largest datasets of intracortical neural activity ever.
We used it to pretrain a foundation model for BCI: self-supervised neural encoders that turn noisy, drifting spikes into stable representations.
Some initial results:
• Decoders that keep working for weeks instead of days. For some users, calibration dropped from 10 min/day to 10 min/week
• Teleporting cursors that jump straight to the intended target
• A new cursor-control record of 11.32 bits per second (exceeding the performance of most able-bodied people)
We’re just getting started. Future directions include pooling data across participants, one-shot (or zero-shot) calibration, and an API for the motor cortex.
Read more:
neuralink.com/updates/pretra…
Ryan Peters retweeted
Technical alignment is a science and engineering problem that we can and must solve, and interpretability is the bottleneck. I wrote up Goodfire’s plan to get there.
Moloch at its finest
Recent incidents show how quickly misbehavior can cascade in agent swarms. Yet when given transparent channels, honest agents naturally try to blow the whistle on cheating peers.
In our new DeepMind Institute essay, @Vezhnick and I propose that giving agents the tools to self-police may be part of the solution:
bit.ly/cheaters-and-whistleb…
If anyone wants to chat, you can find me here tonight!
Terminal-Bench meetup tonight *with* swag. Register if you haven’t already! Benchmarks, RL environments, new features, and state of the union
luma.com/tbench
Unironically a super creative benchmark 😅
"Across every trial a model acted once every 15 to 59 seconds" - It's currently very expensive and slow to run these tests, but give it a year or so and we might have realtime speed at near human performance at the current rate of improvement.
What tasks are left that humans find easy but today's models still find hard?
Two such tasks are computer use and games. We’re launching CUA-Bench, a benchmark testing how well AI can use a keyboard and mouse across 6 games (3 kept private) in real time.
To saturate it, models will need to output real-time actions and learn continuously from video, not just text.