The real world abounds with recursively self-improving systems, but one in particular deserves attention: Science, modeled as a system (perhaps even as an agent, with goals and resources). If you want to really understand AI RSI, science should be your reference point. Science is an intelligent system, and it is obviously recursively self-improving: 1. Scientific discoveries unlock new technology that helps build better experimental tools. This is a top driver of progress in nearly all fields. 2. They unlock new conceptual advances (ideas, theories) that help solve more problems. 3. They increase society's economic output, leading to more resources flowing into science. 4. They unlock better faster tooling (e.g. more compute via better chip & networking technology). As a result, many measures of scientific *input* grow exponentially: 1. Headcount (doubles every ~15 years) 2. Global R&D spending (doubles a bit faster, every ~13 years) 3. Papers and patents (technically this is a measure of headcount) 4. Compute dedicated to science (doubles every ~2 years) But is scientific progress exponential? Historically, the rate of scientific impact over time has remained roughly constant since the start of the industrial revolution (i.e. scientific progress is *linear*). 1850-1900 was about as dramatic as 1900-1950 or 1950-2000. 1850–1900: Evolution, germ theory & antiseptic surgery, thermodynamics, electromagnetic field equations, the periodic table, pharmaceuticals, electricity, telegraph and telephone, internal combustion engine, skyscrapers, mechanized agriculture... 1900–1950: special and general relativity, quantum mechanics, nuclear fission & atomic energy, antibiotics, genetic theory, electronic computers, information theory, synthetic polymers and plastics, the transistor, aviation... 1950–2000: DNA, genetic engineering, integrated circuits, microprocessors & personal computing, the Internet, crewed spaceflight, moon landing, satellite communications & GPS, standard model of particle physics... In real terms, like life expectancy, which has increased in a remarkably linear fashion of roughly 3 months per year since 1840, progress is a straight line. This is especially apparent for fields where impact is easy to measure, like biology, medicine, and agriculture. I first wrote about this phenomenon and its causes in 2012, and a steady stream of research has confirmed it in the years since. Examples include the 2018 paper by Nielsen and Collison, "Science Is Getting Less Bang for Its Buck," and the 2020 economic paper, "Are Ideas Getting Harder to Find?" (In fact, I believe the Nielsen paper stemmed from a conversation I had with him about this exact idea six months earlier) In short, the primary cause is that research solves the highest-impact, easiest problems first, and every subsequent problem is either harder or lower-impact. Exponentially so. The paper that presented information theory wasn't very hard to write (single author!) but you'd have a hard time ever writing a CS paper that beats it in impact. This is why science as a system requires exponential resources (input) to produce linear impact (output). It gets exponentially harder over time. Worth thinking about if you're pondering RSI for AI. I fully believe AI RSI is already happening and will accelerate in the future. But I do not believe this leads to an "intelligence explosion" -- that would fly in the face of everything I know about intelligence and everything I know about recursively self-improving systems.

2022 was a fucking fever dream

It's wild to realize that this was true when it was written four years ago, and that it will never be true again

NEW: According to a bombshell report in the New York Times, Anthropic co-founder Chris Olah threatened to walk out of Pope Leo XIV’s AI encyclical launch in May because the pope rejected the idea that machines can be conscious. Olah’s team then privately lobbied the pope’s advisers “to take the possibility of model consciousness seriously.” Pope Leo XIV held firm. For months, Anthropic has wined and dined theologians and religious scholars under nondisclosure agreements, hoping they would bless the idea that Claude has moral standing. thelettersfromleo.com/p/nyt-…

Timnit Gebru doubles down on the "stochastic parrots" framing, saying you "cannot expect LLMs to be factual." As evidence to support this, she cites errors in... Google AI Overviews. We need to start a GoFundMe to pay for these people to have access to Opus 5.5 and Astra.

We've posted three new misalignment reports. We will keep making those disclosures on a regular basis, independently of whether or not there is an impact on a third party, per our misalignment disclosure framework. alignment.openai.com/misalig…

A core problem with schooling is it forces us to find closure. All topics have to be closed to a form that passes their respective tests. No closure, wrong answer. Nothing in real life is like this. Everything is forever open, changing, adapting, evolving. Get into a relationship with fairy tale ideas of how it's supposed to work -> disaster. Settling on a personal identity and expecting a job to cater to it -> disillusionment. Force every new event into an ideology that explains how society works -> cynicism. Label yourself an introvert, bad at math, not creative, etc. -> adopt a fixed anxiety. Stop trying to find closure in real life. The reality is everything is an open system, and you are meant to thrive within it.

I had assumed it'd be programmers, or maybe lawyers or doctors who'd be first to cry out that they should be protected from AI besieging their unique place in society. Instead it's mathematicians.

This is one of my favorite features of the iPhone Duo. When you’re on FaceTime, you can utilize both screens and cameras. If you’re pointing the camera at someone in front of you, the person on the other side of the phone can talk to whoever you’re on the call with too. You don’t need to flip the phone like we do now. Very impressive software/hardware integration.

Now I can’t keep track of which personal assistant I have doing which task

Can you reverse the teapot spin with your mind?