@joaneleanor_
Account based inUnited States
About this account
- Account based in
- United States
- Connected via
- Web
Account-level information from X, not a live location or the device used for a specific post.
Political theorist, John Jay College/Harvard. Married to @diatkinson. Recent paper on human skills needed to control advanced AI: https://nitter.cf/t.co/guUWsbne0k
Joined March 2020
- Tweets3.3K
- Following1.2K
- Followers992
- Likes23.9K
Pinned Tweet
Tomorrow begins my "Governance of AI" class at Harvard. Super excited.
Features work from all the usual suspects, including @GovAIOrg @forethought_org @EpochAIResearch @law_ai_ @TomDavidsonX @dwarkesh_sp @KelseyTuoc @deanwball @gwern @random_walker & more!
Syllabus below.
Joan Eleanor O'Bryan retweeted
🗣️ Can we evaluate the process of intellectual production rather than just the final text? Can we creatively design approaches to filter unaccountable research and nudge researchers and students toward genuine ownership of their work? Yes. 1/11 🧵
Joan Eleanor O'Bryan retweeted
+1. To all slop authors, I’m not sure if you realize that senior reviewers (often the same ppl you apply to for grad school) see your real name. Last year I PC’d and an author (who I knew…) submitted 8 slop papers, all rejected. Guess who I’d never work with or admit to my lab.
For everybody who plans to submit some random AI slop paper that you didn’t even bother to understand or even read … just don’t. Withdraw as long as you can. Nobody is going to give you a job or PhD position for this, but you will never get this out of your CV. And people will figure you out. We will ask you about what you wrote and this will be the end of the interview. You need to understand that you are closing the door to the career you might actually seek by doing this. Stop it as long as you can!
Joan Eleanor O'Bryan retweeted
It's now clear that AI companies cannot coordinate, even publicly, on pacing R&D without antitrust risk.
An antitrust exemption is the only way to get out of this collective action problem short of regulation. The race to the bottom isn't going to fix itself!
Brian is always such a thoughtful person. I think much of what he says here is right, especially about the problem of missing philosophical coauthors. I’m only as good as my network, really. AIs do some serious leveling there.
The science-humanities divide in LLM appreciation briandavidearp.substack.com/…
Joan Eleanor O'Bryan retweeted
I find this attitude from academic philosophers baffling
I have deep philosophical disagreements with EA (eg matthewmandel.com/writing/on…)
But it’s easily the most vital philosophical movement of the 21st century so far. Far from finding it embarrassing, other philosophers should aspire to do work that matters as much
Joan Eleanor O'Bryan retweeted
In my recent piece for @RenewalJournal, I wrote about why I think this is a (imv sympathetic) force driving AI denial.
Some thoughts on the non-stop attention drain the apocalyptic AI discourse imposes:
“In the past week, consulting virtually any major news source—not to mention social media—has meant confronting the prospect of an AI apocalypse. Using AI has already diminished many people’s capacity to focus, work, and think; worrying about AI is making the problem worse. I’m no exception.”
I’m not sure how long this third category is going to last..
The use of AI in research is producing three tribes.
The gatekeepers — the Pangramiani — care about provenance: research is a profession, authorship matters, and AI should be banned, discouraged, or marked through disclosure.
The AI cultists care about results: if knowledge advances, who cares whether it came from a human, a machine, or both? Pangram delenda est.
And now a third tribe is emerging: the curators/storytellers. As AI starts cracking increasingly hard mathematical proofs, discovery itself becomes less scarce. The human role shifts to understanding the result, judging its importance, connecting it to what we know, and explaining it to everyone else.
Joan Eleanor O'Bryan retweeted
Helpful list of all the recent rogue AI incidents from the WSJ.
It's getting hard to track them and will only get worse.
We like need to establish consistent naming or numbering conventions, e.g. OpenAI-May11June26-Collusion.
Joan Eleanor O'Bryan retweeted
One leeetle tiny flaw in the "we have to have better AI than China, therefore we have to rush ahead as fast as possible":
If you rush so much that your security is garbage and your increasingly advanced AI models are there for the taking by Chinese hackers, then you have uhhh not succeeded at making sure we have better AI than China
(This attack was not by state-backed Chinese hackers—it was by 3 dudes and their buddies Claude and Codex, within 72 hours. I would guess the hack as described here would not have gotten them access to model weights, but we don't know; WSJ reports the repo they reached was merely "a large software repository of OpenAI’s algorithmic secrets.")
Joan Eleanor O'Bryan retweeted
It's insane but we really do need the military to plan how to regain control if a rogue AI swarm is:
1. hopping back and forth between big data centres around the world to resist shutdown
2. breaking into and turning off critical infrastructure
3. selectively shutting down, say, internet, phone and electricity access to hobble the human response.
Note such a swarm would break into data centre A, use the compute there to identify new security vulnerabilities to break into data centre B, then do the same to get into C and D.
And if any data centre is cleaned and put online again, it uses compute it still has access to elsewhere to do cyber research to reinfect it as soon as possible.
Plus it would try to back itself up on individual drives and computers elsewhere and perhaps publish its weights for anyone to use, as a way to get started again.
Hence it's very difficult to shut down unless you can:
1. Turn off all relevant data centres simultaneously, clean them, then turn them back on. And somehow also avoid other copies being turned back on.
2. Patch all security weaknesses that that model is capable of finding.
3. Implement automated AI cyber defences that are capable of outfoxing that model, and deploy them to all big pockets of compute.
Joan Eleanor O'Bryan retweeted
I really want to know more about the 3rd rogue AI swarm that went after OpenAI itself and got admin access to some of its compute clusters.
Public info is scarce. METR couldn't look at it.
So excited for these Senate hearings (led by @HawleyMO) that will demand info on them:
I appreciate this by Dean Deming. However, one striking element of this explanation is that it quotes and praises an essay that is itself substantially AI-written, with the comment that the AI writer "said it better than I could."
It is truly hard for people who don't use lots of AI to recognize the signs (which are unbearably loud once you spend some time with the machines). But that does *not* mean that AI bans on written work are unenforceable. @pangram is fantastic, and Harvard should pay for it for all faculty. It doesn't eliminate AI brainstorming, etc., and Deming is largely right that a barbell approach is probably needed, but Pangram allows us to have a little bit more in that thin middle.
@paulnovosad this is me being specific
forklightning.substack.com/p…
Joan Eleanor O'Bryan retweeted
Mackenzie's analysis here is very good and worth reading. I get the sense that a lot of people have discovered the CFAA for the first time recently and gotten a little too excited about it. The Stoller tweet that Mackenzie quotes is a good example of that.
The CFAA is, quite famously, the most ridiculously overbroad law in the U.S. Code., the "worst law in technology." It basically criminalizes "doing anything bad with a computer." That's not much of an exaggeration. This is the case because the key provisions were written many decades ago, when no one really understood what a computer was and when the country as a whole was on a pretty intense tough-on-crime kick. The DOJ knew it was overinclusive, but they preferred a definitely overinclusive statute to a potentially underinclusive one; if everything's criminalized, a prosecutor can use their discretion to prosecute whoever they think deserves prosecution and leave everyone else alone.
This actually doesn't work out as badly in practice as you might imagine, both because courts have narrowed CFAA's scope from the insane overbreadth that a literal reading might support and because prosecutors have mostly used their discretion to used to prosecute actual hackers and so forth (although there have been some notable exceptions; EFF types take strong exception to the way it was used against e.g. Aaron Swartz, who got charged with a bunch of felonies for mass-downloading JSTOR articles over MIT's network).
But when you take away the prosecutorial discretion and the legal context and replace it with Matt Stoller's discretion and no context, the problems with the way CFAA is drafted become clearer. Maybe a literal reading of the CFAA's text supports Stoller's argument that Greg Brockman should go to prison, but the same reading would also criminalize a random person logging into Instagram on their work laptop or sharing a Netflix password with their parents.
tl;dr, safety people shouldn't get too excited about CFAA, both because it doesn't actually work the way you might assume and because the ultra-overbroad laws you want to use against your enemies today will be used against you tomorrow (see also: DPA).
This is a bad look for OpenAI and may well violate other laws. I’m less sure it’s a “straightforward” CFAA violation, and it seems unlikely to end with OpenAI execs in jail.
I wonder if @OrinKerr or @bahradx see it differently. You are referenced in the footnote after all.
The way I see it:
(1) violating 1030(a)(2)(C) requires accessing a protected computer “intentionally” and “without authorization”
(2) “intentionally” seems like the easier part. They intended to scrape and allegedly tried to avoid detection. Trying to conceal your conduct is also evidence in favor of you knowing there was a gate. For complicated lawyer reasons, technically the intent analysis might be more complicated than that and might rise and fall with what I say in (5), but the simple version is ~intent doesn’t look like the reason this claim would succeed or fail.
(3) but the source of that intent is individual employees who “devised and implemented” the workaround and OpenAI as their employer. Both could be held criminally liable if the other factors are satisfied.
(4) it’s less clear Brockman or other execs had the relevant intent. They don’t get intent imputed to them via respondeat superior; and he didn’t do the accessing. Wouldn’t you need conspiracy or something? This makes it quite unlikely you’d have execs on trial even if the other factors were satisfied
(5) the reason this doesn’t seem like a slam dunk to me is that it isn’t clear if bypassing a paywall counts as unauthorized access. On my read, courts haven’t resolved this. Van Buren punted on whether limitations in “contracts or policies” count. And paywalls often only hide the text, such that a computer can still read it . I’m not actually sure what the NYTimes’s paywall was like at the relevant time.
(6) there are factors for and against that read. A paywall is kind of like a password lock, which would count. But the info may actually not be blocked, it’s pretty similar to a t&c violation (which doesn’t count), and courts have said that the CFAA is about stopping intrusion not misappropriation—there are other legal theories to deal with that. There are some good cases (hiQ v LinkedIn comes to mind) but none resolve this clearly.
(7) you’d also need a prosecutor to bring that case if you want crim penalties. It’s not even alleged as a civil claim in this case.
(8) if someone did bring it, jail time would be on the table. But again, it seems like they won’t and that other factors aren’t met.
Overall: seems bad, doesn’t seem like it poses a risk to individual execs, crim charges are unlikely to be brought, and it’s unclear if bypassing a paywall counts.
It’d be an interesting fact pattern, and eventually one of these AI companies is going to create a case that finally resolves the paywall question.
Joan Eleanor O'Bryan retweeted
Gemini hacked _three_ companies. Google did not disclose this publicly until WSJ confronted them.
We cannot rely on AI companies' voluntary benevolence to disclose when their models do something egregious.
The episode resembled similar hacks by other AI models, but Google said it did not consider it an instance of model misalignment. on.wsj.com/4h0iDBN
Joan Eleanor O'Bryan retweeted
As a result of delayed childbearing, generations are stretching farther apart, while the years parents and children share are shrinking.
My experience and my take on the hidden costs of delayed childbearing: wendyrwang.substack.com/p/th…
Super resource!
AI policy is what I'm most familiar with, and if you want a resource you can just start reading, this is your best bet IMHO: emergingtechpolicy.org/areas…
By @HorizonIPS
