@sidpatl
emulated.so/
Joined June 2026
  • Tweets11
  • Following52
  • Followers23
Sid Patllollu retweeted
Ryan Peters
@ryanirlSep 2
Autoresearch Bench is our first attempt at measuring a model's ability to make meaningful progress on research problems over extended time horizons. Very excited to get this out, and so proud of the hard work from our team!
Joseph Wang
@_josephwang_Sep 2
Announcing Autoresearch Bench A benchmark for coding agents autonomously tackling research problems While existing coding benchmarks are saturating quickly, the difference between models in autoresearch loops is stark
2
6
710
Sid Patllollu retweeted
Joseph Wang
@_josephwang_Sep 2
It was very interesting seeing the different behaviors for long-horizon tasks: - GPT-5.6 and Grok iterated fast & often - Opus liked to make larger, more substantial changes We'll be sharing the 12 full hour runs soon! Certain models even plan according to how much time they're told they have
himanshu
@himanshustwtsSep 2
incredible set of tasks and pretty much long horizon has very long way to go across domains.
2
1
16
1,715
Sid Patllollu retweeted
Utah teapot 🫖
@SkyeSharkieSep 2
This is pretty cool - though I am curious about what happens with Fable 5.1's classifiers on this benchmark due to Ant's previous discussions regarding ML tasks. It seems like their latest classifiers stuff is focused more on bio and cyber sec, but yeah.
Joseph Wang
@_josephwang_Sep 2
Announcing Autoresearch Bench A benchmark for coding agents autonomously tackling research problems While existing coding benchmarks are saturating quickly, the difference between models in autoresearch loops is stark
1
1
10
858
Sid Patllollu retweeted
tokenbender
@tokenbenderSep 2
some really good tasks here. i had an opportunity to look at some of these earlier. benchmaxxing these should lead to better models for sure.
Joseph Wang
@_josephwang_Sep 2
Announcing Autoresearch Bench A benchmark for coding agents autonomously tackling research problems While existing coding benchmarks are saturating quickly, the difference between models in autoresearch loops is stark
2
38
2,098
Sid Patllollu retweeted
himanshu
@himanshustwtsSep 2
incredible set of tasks and pretty much long horizon has very long way to go across domains.
Joseph Wang
@_josephwang_Sep 2
Announcing Autoresearch Bench A benchmark for coding agents autonomously tackling research problems While existing coding benchmarks are saturating quickly, the difference between models in autoresearch loops is stark
5
1
88
8,246
So much fun working on this, proud of the team!
Joseph Wang
@_josephwang_Sep 2
Announcing Autoresearch Bench A benchmark for coding agents autonomously tackling research problems While existing coding benchmarks are saturating quickly, the difference between models in autoresearch loops is stark
1
6
158
Sid Patllollu retweeted
Joseph Wang
@_josephwang_Sep 2
Multiple companies are betting on automating the experimental loop. But which model is the best? - Opus 5 is the best overall - Grok 4.6 is second and efficient
1
4
44
3,203
Sid Patllollu retweeted
Joseph Wang
@_josephwang_Sep 2
Announcing Autoresearch Bench A benchmark for coding agents autonomously tackling research problems While existing coding benchmarks are saturating quickly, the difference between models in autoresearch loops is stark
35
56
13
530
57,234
Sid Patllollu retweeted
Joseph Wang
@_josephwang_Aug 3
At @aiDotEngineer, we spoke about the unique challenges we're tackling at @emulated_ai: - Simulation of infra at scale - Novel research - Sandbox infra for clouds youtube.com/watch?v=zkX03APV…

Emulated: The Data for Fully Autonomous Software Engineers and...

To train an agent that can run production software, you need traini...

youtube.com
1
3
15
3,048
Sid Patllollu retweeted
Chris Barber
@chrisbarberAug 3
i made some updates to pavlov’s list and am pleased with them! thanks to @xeophon @ybenpan @phoebeyao @amit05prakash @kate_shapova @timshi_ai @kevinhou22 @madiator @dlbydq @daljeet_v @ingmariusX @davidstutz92 & Dylan Rogers for feedback lately (pavlov's = list of rl env cos)
4
5
3
47
14,991
Sid Patllollu retweeted
Joseph Wang
@_josephwang_Aug 3
Proud to be featured and have @chrisbarber as our angel! He was one of the first to back our work automating infra and MLE. More to come soon!
Chris Barber
@chrisbarberAug 3
i made some updates to pavlov’s list and am pleased with them! thanks to @xeophon @ybenpan @phoebeyao @amit05prakash @kate_shapova @timshi_ai @kevinhou22 @madiator @dlbydq @daljeet_v @ingmariusX @davidstutz92 & Dylan Rogers for feedback lately (pavlov's = list of rl env cos)
2
20
2,590
Load more