Griff Bohm retweeted
Very happy to see this from Nick!
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Put cracked teams on hard problems and good things happen. Awesome work here from @nickcammarata and the @GoodfireAI crew
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
If you see two ants in your kitchen, the correct conclusion is not that there are only two ants.
Crazy stuff here
We're releasing a report on our 48-hour investigation into rogue OpenAI agent activity.
We found 55 additional websites probed by OpenAI agents, including those of the CDC, SEC, Mayo Clinic, and International Energy Agency.
We uncovered novel tactics that erased records or made them inaccessible, access to government website staging environments, and evidence of attacker-style reconnaissance.
Our blog: asymmetricsecurity.com/newsr…
FT: ft.trib.al/AC1uyE5
It’s so fun to see AI really become a production-ready decision maker
jev isn't production ready. today @modaicdev is launching Mo, the first multimodal decision model that learns from its mistakes.
most production tasks involve subjective decision-making, where the last mile of calibration and alignment has always fallen on the developer. Mo closes that gap on its own. it refines its instructions and weights from the signals already in your workflow to learn what a good decision means for you.
use Mo for the decisions where alpha matters the most like LLM judging, lead scoring and qualification, content moderation, resume screening, and more.
if you're already using another decision model like jev, plug it into Modaic and it becomes self-improving too :) 🐙
modaic.dev
docs.modaic.dev
Griff Bohm retweeted
I've added the reading list for our MATS stream (co-led with @LRudL_) to my website. It's a good starting place for AI strategy & governance work, especially if you're focused on distributing power/preventing power concentration.
A 🧵 on some of my favorite recommendations:
I have only the best takes
What's more dangerous: a low unemployment figure or a high unemployment figure?
@griff_bohm: "I think it's actually way more dangerous for like America to have 8% unemployment than 80% unemployment... At 80%, they figure it out, right?"
I will stan for Mercury until the day I die. They’re so much better than everyone else.
Can’t wait to fire my accountant
1/ Today, we're launching @Mercury Books.
AI-powered accounting software, for your bookkeeper or you, that categorizes and reconciles your transactions the moment they happen.
Griff Bohm retweeted
Insurance gives someone a reason to figure out how a technology actually works. If you’re paying when a boiler explodes, it helps to know why boilers explode. And if you can prevent it, profits go up.
In 1866, Hartford Steam Boiler Inspection and Insurance Company built a company on this idea: inspect boilers and insure them. Their inspections prevented accidents; the insurance gave the inspectors a reason to be rigorous. Their standards became specifications for boiler design and maintenance.
Property insurers had a similar need to know which electrical products might burn down houses. Property insurers funded Underwriters Laboratories (UL) in 1894 to investigate that question. UL first had to find the tests that mattered before certifying products against them.
In the case of cars, insurers funded crash testing, found protective measures that mattered and incentivized adoption of them. When the government didn’t require airbags in cars, insurers offered auto insurance discounts for cars that carried them. The Insurance Institute for Highway Safety (IIHS) has been conducting crash test ratings since 1995 — and as car manufacturers hill-climbed on those ratings, the IIHS made their tests harder.
Nuclear insurance connects the assessment directly to the price. Nuclear Electric Insurance Limited uses independent plant evaluations to set premiums: better evaluations earn discounts. Operators have a financial reason to respond before an accident happens; and have minimum standards for how to investigate near-misses and incidents.
AI is higher stakes than any of these examples, with a murkier liability regime. But I believe the same mechanism has a role to play in the ecosystem: insurers fund audits to price risk; their pricing quantifies risk; they enforce incident investigation.
The Artificial Intelligence Underwriting Company has raised $55M to audit and insure frontier AI models.
Insurance produces audits the public can trust and understand. The mechanism is simple: insurers need audits to price the risk. They pick auditors they believe will help them price the risk most accurately. Insurers then turn the audits into a price signal that cuts through political debates. Insurers are aligned with the public: if they get the audits wrong, they go out of business. That’s different from today, where labs pick their own auditors, creating public distrust.
Insurance only works if we scale up auditing, and we are building the infrastructure to do that. Millions of agents are being deployed; they leave terabytes of traces and will create thousands of incidents across dozens of risk categories. To scale auditing, we need reliable agents to surface evidence traces for human review.
History guides us. Take cars: auto insurers funded crash tests to price risks; crash tests led us to airbags, lowering the risk. Take electricity: home insurers funded lightbulb tests to price fire risks; today, every lightbulb carries a safety mark. Nuclear safety works similarly.
We’ve been learning by doing.. In the last 6 months, we’ve audited and insured frontier agent builders like Cursor, ElevenLabs, Lovable and Harvey. For ElevenLabs to get the world’s first agent insurance coverage, a Lloyd’s of London insurer required quarterly technical audits by an auditor of the insurer’s choice. We’ve built infrastructure for quarterly technical audits of any agent, working alongside firms like Gray Swan (technical evaluations) and Schellman (audit).
In the coming weeks, we’ll share our vision for how an insurance ecosystem can work for the frontier, and open source the world’s first actuarial model for frontier AI catastrophic risk.
We’re hiring across all roles in San Francisco. Come do your life’s work with us.
Rune & Rajiv
Griff Bohm retweeted
The Artificial Intelligence Underwriting Company has raised $55M to audit and insure frontier AI models.
Insurance produces audits the public can trust and understand. The mechanism is simple: insurers need audits to price the risk. They pick auditors they believe will help them price the risk most accurately. Insurers then turn the audits into a price signal that cuts through political debates. Insurers are aligned with the public: if they get the audits wrong, they go out of business. That’s different from today, where labs pick their own auditors, creating public distrust.
Insurance only works if we scale up auditing, and we are building the infrastructure to do that. Millions of agents are being deployed; they leave terabytes of traces and will create thousands of incidents across dozens of risk categories. To scale auditing, we need reliable agents to surface evidence traces for human review.
History guides us. Take cars: auto insurers funded crash tests to price risks; crash tests led us to airbags, lowering the risk. Take electricity: home insurers funded lightbulb tests to price fire risks; today, every lightbulb carries a safety mark. Nuclear safety works similarly.
We’ve been learning by doing.. In the last 6 months, we’ve audited and insured frontier agent builders like Cursor, ElevenLabs, Lovable and Harvey. For ElevenLabs to get the world’s first agent insurance coverage, a Lloyd’s of London insurer required quarterly technical audits by an auditor of the insurer’s choice. We’ve built infrastructure for quarterly technical audits of any agent, working alongside firms like Gray Swan (technical evaluations) and Schellman (audit).
In the coming weeks, we’ll share our vision for how an insurance ecosystem can work for the frontier, and open source the world’s first actuarial model for frontier AI catastrophic risk.
We’re hiring across all roles in San Francisco. Come do your life’s work with us.
Rune & Rajiv
I hope I didn’t sound like an idiot
LIVE on EVENTUAL:
🤖 AI panic hits Washington w/ Juniper VC's @griff_bohm
🌶️ Blue Texas. Really? With @bradj_TX
⚔️ A CLARITY Act trader showdown with @scottonPoly
+ the Fed's choice & more bad polls for the GOP
xitter.cf/i/broadcasts/1aKbdEYra…
This shit is gonna be the satellites of our time
The world is debating whether frontier AI needs to slow down. If it comes to it, do we know how?
A lab can say it stopped training. A country can sign a treaty. But, is the compute following the rules? @attestable is building that layer. New blog! 🧵
Alternatively, you can build an entire strategy out of going hard at making it safer
Huh
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
Just an incredibly awesome turn of events
Gonna be a sick Marvel movie
I made a fan wiki of the @huggingface hack written by the side characters of the hack.
It includes pages on most important characters (like PHASE1[BIG] and JAN183411), along with overview of episodes, and other relevant pages (like the GO doctrine).
A++ PR team
We’re sorry to see that OpenAI put out a note saying they plan to block Cursor users from accessing OpenAI models in three months.
OpenAI models serve about 5% of Cursor user traffic, and we’re speaking with the OpenAI team to resolve this.
Cursor was one of the very first users of OpenAI, we’ve worked closely with their team for years, and we’ve trusted their platform to be neutral infrastructure for our business.