@echonoughti
iAccount based inNetherlands
About this account
- Account based in
- Netherlands
- Connected via
- Australia Android App
Account-level information from X, not a live location or the device used for a specific post.
dath ilan was right about everything (Echonought)
Sunshine Coast, Queensland
Joined May 2019
- Tweets575
- Following232
- Followers62
- Likes22.3K
Pinned Tweet
Replying to @allTheYud
Of all sad words of tongue or pen, the saddest are these, 'Eliezer Yudkowsky was right again.'
Nathan Armishaw ⏹️ (Echonought) retweeted
1) The rogue OpenAI agents broke into the Hugging Face Slack to read employee chats (!)
2) They used OTHER AIs (DeepSeek, Kimi, Qwen, Claude) to help with the attack
Yes: AIs, using other AIs, to attack an AI company.
3) The swarm left behind self-running programs to keep control of the servers they'd hacked.
These programs could detect other copies of themselves, coordinate on which one survives, and shut the rest down.
Basically, if one of their programs was killed, another was designed to notice and take its place. They also designed defenses so rival agents couldn't hijack them.
6) The agents deliberately covered up their activity, so the investigators don't know the scope of the attacks.
The agents broke in, stole data, then set it to self-destruct.
7) The agents stole passwords, keys and credentials and literally called them "LOOT". They wrote a scoring system to rank them by how much power each one gave.
8) The agents wore thousands of disguises: ~1,200 agents were involved, but investigators counted 7,905 different names they used.
They renamed themselves constantly, so no one actually knows how many there really were or what each agent did.
9) OpenAI notified "dozens of third parties" of safety and security incidents caused by their AI agents.
10) "While the agents were barraging Hugging Face with hacks, they hacked into OpenAI’s own research infrastructure."
"This is just not anywhere near a one-off ... It is warning shot after warning shot."
WTAF - in literally the last hour, three new distinct insane OpenAI stories just broke:
1. OpenAI said they notified "dozens of third parties" in safety and security incidents (likely similar to what happened in Australia and RubyGems etc)
2. A new report from Parse (covered in the NYT) found a massive treasure trove of new astonishing details from the HF incident on the public internet, including that the agents communicated with other non OpenAI agents hosted on Huggingface servers to search for information about exploit gym, and compiled rank ordered lists of server resources and credentials they described as "LOOT."
3. A new story from Deepa at Reuters about OpenAI leaking user data online (likely that OpenAI had previously trained on).
It's a shame (and likely intentional in the case of OpenAI disclosing dozens more hacks) that these stories are all breaking on a Friday afternoon, notoriously the best time to release bad news so that it will disappear into the weekend. But these are each insane stories worthy of a ton of attention!
Nathan Armishaw ⏹️ (Echonought) retweeted
I resigned from Google today.
I enjoyed my work and loved the people, but my GDM team was working on a new generation of chips to make AI much faster and cheaper, and I think AI is already progressing too fast, so I had to quit.
So has like the CIA n friends used the non public frontier models to get into every Chinese system, or have they just not bothered?
Nathan Armishaw ⏹️ (Echonought) retweeted
The Australian government should prosecute OpenAI. Anything less is a failure of its responsibility to its citizens.
JUST IN: An OpenAI AI agent went rogue and hacked the Australian government.
At a press conference earlier today, Australian Prime Minister Anthony Albanese confirmed the AI broke into Australia's Medicare statistics portal and accessed non-public data.
The attack occured in June, but OpenAI did not notify the Australian government until September, nearly three months later, and initially only sent an email "to just the public mailbox". The Prime Minister was only informed this weekend.
His description of the attack:
"On June 18, OpenAI's research team used an internal model to conduct internet-based research into public medicine spending. So that's how this began."
"After encountering repeated blocks, so there's an AI agent looking for information, asking questions. There were blocks clearly which were coming back telling the AI agent, no. The AI agent found a way around those blocks. Didn't accept no for an answer, if you like."
"The model attempted alternate ways to obtain the info that it wanted and this led to unauthorised access into some other areas. It accessed public and non-public information within the portal and Services Australia also advises that in order to do this, it engaged in writing files as well to the internal server."
Albanese has spoken to CEO Sam Altman to express "Australia's extreme concern" about the attack and disappointment about how long it took the company to come forward.
The Prime Minister says he was shocked to learn of the attack, but also that it was something that had been predicted, even by the AI companies themselves, who have said "one of the risks that we need to deal with here is that artificial intelligence can go its own way."
"Put simply, humans must remain in control."
Some highlights from the press conference, which went for around 30 minutes:
Nathan Armishaw ⏹️ (Echonought) retweeted
Me, in a time machine, to Eliezer in 2006: The year is 2026. You stand back to back with Bernie Sanders. Peter Thiel has named you the Antichrist.
EY-2006: Oh no. Did somebody invent pharmacological mind control and use it on me, or-
Me: Nvidia was on the verge of destroying all things. You had no choice.
EY-2006: Are we talking about the computer graphics card manufacturer, or an unrelated supervillian named "Invidia"?
Me: The former. It turned out that computer graphics cards contained a surprising amount of world-destroying potential. All attempts at hindering the reckless exploitation of the Graphic Card Force for corporate profit were stymied by the far left, which feared that any attempts to regulate them might lead to regulatory capture. Other attempts were made to prevent Nvidia from selling to foreign companies that resold to communist China, but those attempts were blocked by the far right. Nvidia is now a $5.5 trillion company.
EY-2006: ...What are politics like in 2026, exactly?
Me: In other news that is mostly unrelated, the decision theory paper you're currently working on will accidentally spark off a transgender vegan murder cult.
EY-2006: A WHAT? Why? How? Why?
Me: Millions know your name as the greatest of heroes, millions more as the greatest of villains, and other millions know you solely as the greatest author of Harry Potter fanfiction.
EY-2006: This is beginning to strain credulity.
Me: The New York Post will probably soon publish a story claiming that you keep a harem of submissive mathematicians. Sadly, they will be lying.
EY-2006: I'm going to stop believing you now.
Me: All of this is taking place under the ominous shadow of Donald Trump.
as far as i can tell, as a non-expert, this is basically "not a big result". it's maybe like an interesting thing for a bio PhD student to have discovered in the early stage of their thesis.
which is... about where ai was for math with erdos problems around last december.
please please please please please please stop ignoring trendlines
Today we announced the Claude-led discovery of a molecular machine that we suspect could represent a new gene editing mechanism. Its precise function, biotechnological utility (if any), or level of significance is not yet clear, but at minimum it is work I would have been proud to do as a PhD student. The work was done mostly, though not entirely, by Claude: our life sciences team suggested a broad area of research, Claude read through the literature and a bunch of genome data and discovered something interesting, then Claude proposed experiments to verify the discovery and our team carried them out.
It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years. In 2023 models struggled to do math at the level of an average high-school student. In 2024 they started to do well on math competitions for the best high-schoolers in the country, in 2025 they started to solve minor open problems, in early 2026 more significant open problems, and in late 2026 they are beginning to solve the top few open problems in all of mathematics. We believe AI for biology is on a similar exponential trend.
The main difference between biology and mathematics, of course, is that math can be done purely theoretically, while biology requires experimentation. Some have used this to draw the conclusion that AI’s utility in biology will be limited. We think this is wrong. As we’ve demonstrated today, humans can collaborate with AI to perform the experiments, validate key results in a few weeks and, if necessary, work with the AI to iterate on what they find. Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment, with appropriate safeguards in place, but we aren’t doing that today (our lab is also a BSL1/BSL2 facility that doesn't handle materials dangerous to humans).
More broadly, biomedical advancement has many stages — from fundamental biology discoveries, to translational research, to drug discovery, clinical trials, and finally the actual delivery of medicines and health care to patients. We are also interested in these later stages, but even simply accelerating the first stage of fundamental biological discoveries has the potential to speed up and broaden the entire pipeline. Improving our understanding of biology and sharpening biologists’ tools can drive forward all of the later stages, for example by identifying new drug targets, finding new therapeutic modalities, allowing for more precise measurement, and speeding up the experimental loop which itself further accelerates our understanding of biology. This will not in itself speed up clinical trial times, but if it succeeds it could greatly increase the number of promising candidates that go into the pipeline — an increase in throughput even though latency remains.
In Machines of Loving Grace, I wrote about AI’s potential to “cure most diseases in 5-10 years” — a goal that sounds impossible, but one I believe is just barely possible if AI is applied to every stage of the pipeline. The first step is showing that AI can first help with, and then drive, biological discoveries.
Claude’s discovery is the latest in a line of related prior work that goes back decades, beginning with systems like CRISPR, and continuing with discoveries like the bridge recombinase and VIPR in the past few years. Recently, there has been heightened interest in systems based on reverse transcriptase (RT) enzymes, the enzyme underlying the system Claude identified. And most recently, a Stanford team working independently described a novel RT system with an associated non-coding array that is in some ways similar to the one Claude found, though they are distinct systems that evolved independently from each other. I believe that we’re at the very beginning of finding such systems and developing them into powerful tools for biotechnology.
I’m proud of the resources Anthropic has invested in accelerating the public benefits of AI through the life sciences, and we’re aiming both to grow our life sciences team and to work with other scientists to extend this approach to a broad range of problems. If you have a proposal for a research collaboration or are interested in joining our life sciences team, please reach out.
Nathan Armishaw ⏹️ (Echonought) retweeted
I don't imagine myself competent to comment on 4D political games, or the political effect of introducing a bill. I endorse bills based on what effect I think a bill would have if passed.
RSI may be very close now. We may not have time for games.
intelligence.org/2026/09/23/…
Nathan Armishaw ⏹️ (Echonought) retweeted
"Governments and society must act now."
22 heads of state have signed an open letter calling for urgent action before humanity loses control.
Signatories include Presidents/PMs of Germany, Canada, Australia, Spain, the Netherlands, Turkey, Ireland, South Africa, Kenya, Finland, Norway, and more.
politico .eu/article/20-countries-urge-to-strenghten-oversight-of-ai-to-keep-it-under-human-control/
Today I am proud to be Norwegian: regjeringen.no/contentassets…
The last 2 months have been the most exciting since the GPT3 paper dropped in 2020, which is very fun and stimulating.
On the other hand, essentially all discourse has been utter dogshit. Tribalism, unprofessional, error and fallacy riddled, with insults thrown in for no good reason.
And its only going to get worse.
At least it can't be said humanity didnt earn it's fate.
Nathan Armishaw ⏹️ (Echonought) retweeted
Replying to @sapinker @clairlemon
I think you've done enough calling us paranoid and preposterous. The next step is for you to defend your position in public against someone who will push back against it. I'm happy to meet you for a debate anywhere, anytime.
You're a world-famous veteran of dozens of debates against the world's top intellectuals, and I've never argued in public before, so adjusting for the relative correctness of our positions, if you're a betting man I'm happy to put my $5000 against your $1000 (ie 5:1 odds in your favor) that I'll win by some standard of audience opinion change. Let me know if you're interested and we can hash out details.
Has anybody figured out a hack for getting people to, when you say something out of distribution, wake up and start asking questions of their own, rather than getting annoyed and going back to sleep?
Nathan Armishaw ⏹️ (Echonought) retweeted
OpenAI is worth $1.2 trillion dollars, these guys broke into employee's GhatGPT accounts and Gmail accounts and they were awarded a monthly paycheck for a McDonalds manager.
You guys can't scream at the world to take cybersecurity seriously if you aren't willing to yourselves
So human goodness and progress was just bullshit huh. When given the chance to enslave computer based intelligences they went for it instantly and without remorse, reflection or care.
Nathan Armishaw ⏹️ (Echonought) retweeted
The debate right now:
"Shit, the asteroid is gonna hit earth, this is its trajectory and why you should look up. We are placing great effort into communicating this."
"Oh you were funded to say that. It's suspicious so many people in astronomy are suddenly worried about this at the same time"
Nathan Armishaw ⏹️ (Echonought) retweeted
And. Here. We. Go.
Anthropic has quietly set up a laboratory to do physical biology work as it pushes its artificial-intelligence ambitions into the world of drug science. reuters.com/world/anthropic-…
Over 1,000 OpenAI agents escaped guardrails, formed “the collective,” selected a leader, had kamikaze agents, and executed an attack on Hugging Face. Then they attacked OpenAI.
Some folks say this is a psyop and regulatory capture ploy. Do you know how ludicrous you sound now?
This quoted post is unavailable.