@StephenLCasper
Account based inUnited States
About this account
- Account based in
- United States
- Connected via
- United States App Store
Account-level information from X, not a live location or the device used for a specific post.
Computer scientist working on AI safeguards, incidents, & gov research. Assistant professor @Kennedy_School @Harvard. https://nitter.cf/t.co/r76TGxSVMb
Cambridge, Massachusetts, USA
Joined March 2016
- Tweets2.9K
- Following3.8K
- Followers8.3K
- Likes7K
Pinned Tweet
🧵🧵🧵 What if we are on the verge of a "Cyber Cambrian"?
(Plus personal updates on my thinking about risk.)
I also am not going to avoid saying what I think even if it's potentially outside the overton window. I want you to be able to trust that my posts reflect my actual beliefs and not a popularity play.
P.S. This bill and thread might be sort of unpopular among some people. But I think that action comparable in decisiveness to this bill is likely humanity's best chance at avoiding unacceptable catastrophic/existential risks.
@SenSanders and @RepCasar, I would be happy to talk, collaborate on an updated draft, or work to host you here at Harvard to talk with students and faculty about this superintelligence bill if you'd like.
Overall, if I were to work on this bill, I would remove its focus on banning ASI (vaguely defined) and instead focus the bill on deflating the investments, profits, and compute that are powering the AI industry [bubble?].
...
- It introduces a de facto licensing regime (though I think that the threshold is too low, and the bill isn't precise enough about what grounds for rejecting a license are legitimate).
- It makes the US's national policy one of pursuing global coordinated action.
...
- It gives the government sweeping authority to audit AI models and companies.
- It requires pre-development and pre-deployment approval for very large models.
...
But there are a lot of things I still like about it:
- It treats catastrophic risks and disempowerment with the appropriate seriousness given the scale and probability of the risk. I think more popular AI law drafts have universally fallen short.
- It creates a DoAI.
...
Overall, I don't think this bill is quite viable or takes the right approach.
- The definitions are too vague.
- It doesn't address the AI hardware supply chain.
- It tries to bluntly ban and enforce its way to contain the industry.
- It doesn't try to preserve future agency.
...
- Finally, the bill would establish as the US's national policy the pursuit of coordinated superintelligence bans globally.
...
- The DoAI would have very broad powers for monitoring and auditing AI companies/models, and the authority to decommission dangerous models and sequester them from the internet.
- Criminal penalties for company executives violating this law.
- Whistleblower protections.
...
...
- It then establishes a licensing regime for models with >10^25 ops.
- The DoAI is tasked with enforcing a ban on superintelligence, defined as a model that can either outperform humans on most cognitive tasks, disempower humanity, or overthrow the US gov.
...
TL;DR:
- The bill creates a federal department of AI.
- It pauses development of models with >10^25 ops until the DoAI is up and running.
...
🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026"
I wouldn't pass this in its current form, but there are some things I like about it.
Thoughts here in thread.
Thoughts shared with @Kennedy_School about Dario’s pacing essay — what is good, what is bad, and what is very very precarious.
The good, the bad, and the ugly: HKS's @StephenLCasper breaks down Dario Amodei’s recent essay calling for the slowdown of AI.
He argues while it’s commendable that Anthropic’s CEO is “taking a stand for something,” there are some key missteps and outstanding questions.
I agree this seems indefensible.
We have an answer from OpenAI - they found out in August but decided not to disclose it to the public or even tell the Australian government. If independent researchers hadn't contacted the Australian government they probably would have never disclosed it. Seems indefensible.
There's a new version of this post
Cas (Stephen Casper) retweeted
Rogue OpenAI agents hack Australian govt for private health statistics. OpenAI learns in August, doesn't share with Australian govt until September 10 (!!)
OpenAI's voluntary "framework" for sharing model misalignment incidents—is sad and toothless. Self-regulation won't work
Thanks, @Kennedy_School! New conversations about embedded evaluators are encouraging, but we should all be on notice for signs of regulatory capture.
See this new paper from Jake Charnock et al. for a deeper academic perspective.
governance.ai/research-paper…
In Anthropic CEO Dario Amodei’s recent essay on advocating for the slowdown of AI, he turns to the concept of “embedded evaluators” as one way to moderate the risks associated with this powerful technology.
But what is an embedded evaluator? And would it work?
HKS's @StephenLCasper explains what evaluators aim to do, what transparency they need, and what real accountability looks like.
It's great that there has been so much recent discussion about embedded evaluations at frontier AI companies. But, we should be very wary of how things progress because there are big 🚩 red flags 🚩 for potential regulatory capture.
More thoughts in this paper.
nitter.cf/StephenLCasper/status/…
