
Researcher at @openai
San Francisco, CA
Joined August 2017
- Tweets264
- Following746
- Followers4.1K
- Likes1.8K
Pinned Tweet
We launched ChatGPT Agent today!
When tested on a variety of REAL work tasks (expert tasks that might take >10h), we found that its output was human-quality almost 50% of the time
Agent puts o3's intelligence into practice - try your work tasks and let us know how it goes!
Casey Chu retweeted
Last year I posted a video about a ~200,000 year old denisovan genome.
Today, we got a pre-print… and some beautiful figures.
It’s seeming more and more likely that our divergences have been underestimated due to the hybridization events that occurred after.
Casey Chu retweeted
It’s time for vibe-lifeing
Casey Chu retweeted
To summarize this week:
- we released general purpose computer using agent
- got beaten by a single human in atcoder heuristics competition
- solved 5/6 new IMO problems with natural language proofs
All of those are based on the same single reinforcement learning system
Casey Chu retweeted
watching chatgpt agent use a computer to do complex tasks has been a real "feel the agi" moment for me; something about seeing the computer think, plan, and execute hits different.
Huge thanks to @gracejkim9 Elizabeth Proehl @michelelwang @marwan_aljubeh @rachelds__ @tejalpatwardhan for putting this eval together, letting us measure Agent's capabilities in realistic work settings 💪
working on bringing that pass@16 number down to pass@1 💪
Replying to @EpochAIResearch @OpenAI
We also found that, when allowed 16 tries per problem, ChatGPT agent’s score grew from 27% to 49% on the tier 1-3 set. This suggests that better prompting or scaffolding might result in better performance from current models.
Great post from @xikun_zhang_, who did a great job making sure collaboration with Agent feels good!
Just launched ChatGPT Agent (sorry GPT-5 waiters, it is coming!), the most capable AI agent model to date! It has been such an honor to be part of a crazy sprint to get this amazing model trained and shipped together with an absolutely gem team (@isafulf , @caseychu9 , @EdwardSun0909 , @josh_tobin_ Yash Kumar and many more)!
I am so proud of this project, so I want to share some highlights, personal takes and lessons learned while working on it:
1. Used for research 📕 + actions 💻 + slides generation: Deep Research can do research. Operator can take actions for you. ChatGPT Agent can do both at the same time! E.g. you can ask it to make a plan for a trip to Hawaii, find good deals on hotels and flights, and book them on your behalf using its own computer! It can also generate slides!
2. Power of end-to-end RL: How do we build it? You guess it right! It is us,
@OpenAI RL diehards. You are probably tired of hearing about RL scaling. Me, too. But when I feel its power first-hand, its effectiveness and data efficiency still shock me and feel like magic 🪄.
3. First OpenAI model of high biorisk 💀: Not sure this is something I should proud of or not :) For an ex-AI bio PhD researcher like me, this is something a bit personal. One one hand, many of my biomedicine researcher friends tell me that AI agents have significantly helped with their research. On the other hand, such a capable model can amplify the risk of malicious actors building bioweapons. Our safety team has done incredible work to mitigate the risks.
4. Collaboration with users 👪 is core: We want our AI to augment and enhance humans, not to replace them, so we work hard to make the model good at collaborating with the user. You can type a message at anytime to interrupt it and steer it to new directions. The model will always confirm with you before taking actions like buying things for you or deleting a file on your google drive. And the model will ask clarification questions only when it needs more clarity from you!
5. How to generate good slides: As in other cases, writing a well-specified prompt always helps! Also try first telling it to generate a report, then convert the report into slides!
6. Real-world performance > benchmark chasing: One thing outside people may not know about us is how little attention we pay to external benchmarks during the model dev process. We do not focus on hill-climbing on them, and we do not care that much about how we end up on the leaderboard. That said, as a byproduct of our pursuit to great real-world performance and true intelligence, ChatGPT Agent does crush many benchmarks! Wanna learn more? Read our blog linked in the end!
In the end, I want to shout out to my amazing team again. These extremely talented and kind people are the reason why OpenAI is constantly making magic like this! ❤️ Also please try ChatGPT Agent and give us feedback! You can reply here in the thread or my DM is open. This is just the start. We will continue working hard towards more and more capable super-human AI agents! 🤖
openai.com/index/introducing…
Join us in making the next generation of agents both capable and safe! We think that agents will be a big part of how we interact with AI in the future, making it critical that we think carefully about how we build them.
We're hiring for a new team @OpenAI: Agent Robustness and Control
Our goal is to make sure our agents safe and secure during training and deployment. Want to work on some of the hardest problems in AI today? Apply via link in reply or DM me!
Casey Chu retweeted
It's deeply concerning that one of the best AI researchers I've worked with, @kaicathyc, was denied a U.S. green card today. A Canadian who's lived and contributed here for 12 years now has to leave. We’re risking America’s AI leadership when we turn away talent like this.
Casey Chu retweeted
LLMs have complex joint beliefs about all sorts of quantities. And my postdoc @jamesrequeima visualized them! In this thread we show LLM predictive distributions conditioned on data and free-form text.
LLMs pick up on all kinds of subtle and unusual structure: 🧵
We launched a research preview of Operator today! It's a model built on top of GPT-4o that can control a browser — it is very early and will make mistakes, but it's a taste of things to come openai.com/index/introducing…
Casey Chu retweeted
Today OpenAI announced o3, its next-gen reasoning model. We've worked with OpenAI to test it on ARC-AGI, and we believe it represents a significant breakthrough in getting AI to adapt to novel tasks.
It scores 75.7% on the semi-private eval in low-compute mode (for $20 per task in compute ) and 87.5% in high-compute mode (thousands of $ per task). It's very expensive, but it's not just brute -- these capabilities are new territory and they demand serious scientific attention.
Casey Chu retweeted
Why don't we measure probabilities in degrees?
blog.alexalemi.com/a-degree-…
Casey Chu retweeted
An intuition for relative memory access times (scaled 10^10):
Reg: 2 sec - Take from shelf
Cache: 6½ min - Get from garage
DDR Main: 20 min - Go to store
DDR CXL: 1hr
Far Mem: 8hr
SSD: 6 days - Order online
Spinning Disk (3ms): 1yr!
Via @dylan522p & @SemiAnalysis_
Casey Chu retweeted
I had the joy and the honor of being invited to give the @harveymudd commencement address this year. In the vector space of all advice, I explore a 5-dimension subspace orthogonal to the “follow your dreams” vector.
YouTube Link: youtu.be/W3I3kAg2J7w



























