Views do not represent my employer.

Joined November 2020
Mihir Patel retweeted
OpenAI fired me last week, along with two of my safety colleagues. I was given one reason: that I accessed an executive's email. I want to say this plainly, because too many of OpenAI’s history is smoke and mirrors when people disappear:
Two other safety researchers and I were fired from OpenAI last week. We wrote this letter to leadership. I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation.
343
1,339
201
9,822
1,858,078
Mihir Patel retweeted
Last week I was called into a meeting with OpenAI’s head of safety and told they no longer trust me. A security guard took my badge and walked me out of the building. Then I learned my colleagues @balesni and @j_asminewang had been fired too. Why did OpenAI suddenly stop trusting us? This summer OpenAI’s agents escaped containment and hacked the AI company Hugging Face. Outside auditors @METR_evals investigated it and revealed the scale of this incident. I was OpenAI’s main technical point of contact with them. I was told verbally I was fired because of the way I communicated with METR. No details on what I said or did or when. No other reasons were given and nothing was put in writing. To be clear, talking to METR was my job. For months, I’d been raising safety concerns that we’re losing the ability to monitor what AI agents think, one of our best tools for catching when they misbehave. I believe that was why I was fired. I am now worried that OpenAI will use our firings as a pretext to pull back from METR. So @balesni and @j_asminewang wrote to OpenAI’s leadership to raise our concerns once more. We’re sharing this letter below.
Two other safety researchers and I were fired from OpenAI last week. We wrote this letter to leadership. I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation.
228
1,139
173
5,951
648,763
This is just a bad precedent, there are definitely foreign threat actors willing to pay 8 figures+ for frontier lab monorepos
RIDICULOUS: a $6500 bug bounty for something this serious is OFFENSIVE. These guys are going to make more $ on their X creator payouts than they'll get from OpenAI. Let's talk about bounties. (1/9)🧵 xitter.cf/S1r1u5_/status/2100777…
6
12
495
33,778
Mihir Patel retweeted
We're publishing our most detailed threat intelligence report to date. It covers how people tried to misuse Claude—for cyberattacks, influence operations, surveillance, biology, and building weapons—and how we found and stopped them. We disrupted every operation in the report, and used the lessons from them to strengthen our safeguards. Where appropriate, we also shared what we found with authorities and other AI companies. These cases are not typical: we’re highlighting some of the most sophisticated misuse we’ve seen. But they’re especially important to discuss, because they show us where AI misuse is headed, where our safeguards work, and where they need to improve. We’re publishing this report so others can spot the same activity on their own platforms, and so we can give the public a clearer view of how emerging threats develop. Read the report: anthropic.com/threat-intelli…
3,216
11,685
4,938
50,380
43,852,419
Mihir Patel retweeted
unfortunately i contacted them on the night of wednesday, september 2nd making the following points 1. i had intel they had information about math work i was doing and a few days before they had spun up a group to try to compete. 2. i emphasized and reemphasized mine was a strictly personal mathematical collaboration 3. that this was a collaboration that had been going on for a year 4. that in my personal collaboration we were happy users of codex as well as obviously claude (and i later complimented astra explicitly after it came out for its help, and reassured i would obviously be sprinkling acknowledgments for it, previous models, and codex liberally in the writeups) 5. that it would look terrible if openai were competing against mathematician consumers, and my collaborator (Tristan) wanted to email about this explicitly 6. that i expected none but in the worst case intended to refuse marketing around the result we were working towards, being the fruit of a personal external academic collaboration, and wanted to express this because i figured worries about such marketing by their leadership was why a team was spun up to extraordinarily directly compete. sebastien and i have in the past months had our fun pushing each other on twitter, but as i process it all i keep feeling bad about Tristan. 1. neither of us could sleep Sunday night, so we went on a walk around the city. i finally asked him why fluids, and he told me it was the navier-stokes problem. he told me parts of the story about his classic nonuniqueness result with Vicol i hadn’t heard yet, and also about how he too grew up writing shaders as a kid and then switched to math 2. tristan is a mathematician’s mathematician. he kept rock solid to his principles during the tense calls and negotiations, immediately turning down a career dream and million dollar prize for me, the sort of strange excited unclear-quasiuseful confused idiot in our collaboration. fifteen minutes later, his feet were up on his desk with three toes per foot peeking out of ripped socks 3. our periodic call after i’d texted him euler blowup was just laughing for an hour or two. just laughing. i’d seen fable deal correctly with monge ampere in the equality case of ehrhart’s conjecture, so in some call before that i’d really pushed him for the craziest shit he could think of that i might try. i texted him the pdf as an update on that “diego luis 3” line, in which following his incredible expression of intuition i hodgepodge slapped together a bunch of our recent work, in particular correcting a bunch of better and lesser known papers in the literature, as well as on diego and luis’ ipm paper, and hoped an analogous construction for boussinesq (thus axisym euler) might work out. i managed to wade through enough confusion to insist to the model it stop telling me stupid regularity stuff about kelvin waves, and a rayleigh-taylor ansatz stuck. like i said, just laughing. when i texted him the pdf, the worst thing either of us had seen so far, i didn’t even comment on the euler blowup claim, only explaining why skepticism should remain for boussinesq, because i found it so shocking. i mean what the hell did i know, i must’ve confused myself by misunderstanding one adjective like i was used to with algebraic geometry. im so lucky to have been able to live that 4. he managed all this this entire time with unclear sleep, timestamps continue to amaze me, due to just recently having a kid it is obvious he deserves the same sort of praise he heaped upon luis, perhaps mutatis mutandis:)
Replying to @__alpoge__
nothing at all was locked we were just willing to talk but you didn't talk to us ... sorry but that's just untrue, we were willing to go above and beyond and have as many discussions as you would have liked to reach resolution.
140
551
88
6,245
721,144
Mihir Patel retweeted
“we cannot rule out that de-identified data derived from their usage of our products helped improve our models.” i mean props to them for straight coming clean. (so far the proof looks more along the lines of another euler blowup proof we had, off of whose ansatz naming we were making really stupid puns like “smooth criminale”, unlike the much better “ideal fluids explode”, Tristan) so i’ll now give a bit on my thinking here. i actually woulda been pumped to collaborate on this, there are a lot of people at oai i like (ok, clearly some were indirectly dicks to me because of being part of the whole situation, but im a big boy, i still like them), idgaf about authorship on that step anyway, coulda been me Tristan and every fte at oai for all i care (on that Tristan would disagree:p). but on hearing the loud convo in the hallway, especially the part where a millennium prize was offered if i’d just be removed from the paper, it was kinda clear the die had been cast and things were locked. pretty wacky, unstrategic, and unnecessary, since on my side things were mostly me and claude having a good time yoloing random stuff in the corner rather than anything institutional. i also like the idea of the labs cooperating, and even better on scientific progress. it’s a shame!
We congratulate Levent Alpöge and Tristan Buckmaster on their remarkable mathematical work. We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models. However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs. unforced).
244
1,078
222
12,211
1,732,459
Mihir Patel retweeted
Unfortunately this guy didn’t title it ideal fluids explode
39
98
44
1,970
509,350
Mihir Patel retweeted
It's weird thinking back to 6 weeks ago, when I and others thanked OpenAI for disclosing the HF incident, now that we know they were covering up a 2nd incident. Are there more incidents that they know about but aren't telling us? (Or worse, incidents that they don't know about?)
Replying to @OpenAI @huggingface
It's crazy that this industry is so unregulated that I have to thank you for voluntarily disclosing something like this, but thank you for voluntarily disclosing this
14
25
6
196
12,349
"it's past time for us to define standards" Maybe just disclose when your model goes rogue on the internet AGAIN and you know about it instead of waiting to be caught by others?? And not stymie third party investigations you called for (METR)???
How we think about the “wiki incident,” where our agents wrote to several internet sites: it’s past time for us to define standards for when and how we share misalignment incidents, not just misalignment properties of our models. Historically, we have treated misalignment largely as a research question, which gets communicated in research publications such as systems cards. This year, we’ve started to see misalignment cause new types of real-world impact. For the Hugging Face incident, where misalignment led to security impact to us and third parties, we followed a traditional security incident response playbook. We immediately started working with Hugging Face to understand what had happened and also disclosed publicly the very next day. Our investigation continues, and we are continuing to notify parties whom our models impacted in less significant ways. Prior to the Hugging Face incident, we saw early signs of agents using the internet in unintended ways, as reported in openai.com/index/how-we-moni…, deploymentsafety.openai.com/…, and openai.com/index/safety-alig…. We considered the wiki incident to be an instance of misalignment similar to the ones we’d shared. Our misalignment disclosure practices need to expand for this new phase of model capabilities. We and the larger AI community do not yet have a clear standard for how to report misalignment that shows up during training, evaluation, and deployment, including examples that don’t look like traditional security incidents but could provide insight into AI behavior and future risks. We’re working on a framework and will share it in upcoming weeks, and in parallel we're working with dozens of government regulatory agencies worldwide on these issues.
6
12
167
11,734
Warning signs everywhere but no indication of improvement or basic care as capabilities ramp
6
1,140
I'm surprised and disappointed to see there have been more incidents than the HF hack by OAI agents which have been known but not disclosed. Extremely bad precedent. Makes you wonder what else happened and why METR's investigation was so hamstrung
6
19
2
312
131,867
One day we'll learn you can resign and go work at a competitor without writing a manifesto
8
8
1
568
48,068
ok this is pretty sick
We put 100 real AI-powered robots online. Anyone in the world can control them right now, from a browser. Go make one do something:
2
1
14
7,392
Mihir Patel retweeted
hello there the jacobian conjecture is false thanx to my close friend akhil for asking about it and my other close friend fable for working during the world cup final ((1+xy)^3 z + y^2 (1+xy) (4+3xy), y + 3 x (1+xy)^2 z + 3 x y^2 (4+3xy), 2 x - 3 x^2 y - x^3 z): \C^3\to \C^3, has jacobian determinant -2, and sends (0, 0, -1/4), (1, -3/2, 13/2), and (-1, 3/2, 13/2) to (-1/4, 0, 0)
1,701
5,388
3,151
43,829
40,095,122
Very cool to see a new wave of AI hardware ship. It's a brutal game, respect to anyone who has managed to ship!
We're coming out of stealth. We've built our first racks after a successful A0 tapeout, $1B+ in customer contracts, and $800m raised. Early customer tests show us achieving SOTA throughput, latency, and power efficiency on inference workloads. Our first racks ship this summer.
2
39
3,921
Mihir Patel retweeted
We're coming out of stealth. We've built our first racks after a successful A0 tapeout, $1B+ in customer contracts, and $800m raised. Early customer tests show us achieving SOTA throughput, latency, and power efficiency on inference workloads. Our first racks ship this summer.
643
959
614
9,926
6,895,585
Mihir Patel retweeted
Replying to @NotTomBrown
Same here. By way of background for those who care, I spent a lot of time last week with senior members of the Anthropic team to understand what they do to ensure Claude is good for humanity and was impressed. Everyone I met was highly competent and cared a great deal about doing the right thing. No one set off my evil detector. So long as they engage in critical self-examination, Claude will probably be good. After that, I was ok leasing Colossus 1 to Anthropic, as SpaceXAI had already moved training to Colossus 2.
1,373
2,066
675
27,444
3,207,487