Brian Johnson

I’m no expert, but can’t they have the all powerful frontier models help catch the Binance, and other such, hackers? What am I missing? They’ve been showing off their power by crushing the dreams of math geeks, but wouldn’t it be more effective to do something useful like protect exchanges and banks?

ASI would require architectural breakthroughs, I think, in LLM design. Not making them much bigger or more efficient. Even via RSI. But who knows? But major architectural breakthroughs will happen. It’s insane we’re just going to sit around and wait for it to be too late. We have deeply delusional leaders! They think, like in a bad movie, that they can summon the demon to do their bidding. When everyone knows the demon always kills the idiots who summoned it first!

There’s lots of discussion these days, much of it dismissive, about whether AI thinks. They’re conducting trillions of computations directed towards solving a problem using a global information processing algorithm of astronomical complexity. What else would you call it?

Physics isn’t my area of expertise but I find the idea that modern nuclear reactors are more dangerous than climate change to be insane. Many on the left and the right have non scientific beliefs whenever it suits them.

The idea that UBI, a welfare payment for everyone not rich when AI takes all our jobs, is going to lead to a stable society is bonkers. What do they really have planned?

It might turn out that the worst thing that ever happened to humanity is the fact that AI is nowhere near as hard to create as we always thought it would be. For ages scientists have considered it a kind of holy grail we’ll get to in the distant future but here it is all of sudden way before we’re ready.

The most powerful technology in all of human history, thinking machines, already doing math at the highest level, have literally no regulations! Try to wrap your mind around that.

Isn’t it interesting that so many on the right, who traditionally love mocking ideologies of social utopia from the left, are 100% on board for a tech AI driven future of abundance where we all live forever like kings and queens?

Most scientists, particularly biologists, still have their heads in the sand about where AI is taking our field. Biology labs are pretty small places employing relatively few highly trained people. This is particularly true in academia but it’s also basically true in bigger specialized industry labs. When AI is used to automate bench work, and I mean all of it, they are not keeping this model for long. They will not be building automated bio labs but rather biology factories. Because with robots doing all the work there is no obvious size constraint. People keep saying but experiments can’t go any faster, it’s physics that determines how fast they go, etc. A bio factory that produces 10-1000x the data which is analyzed as it’s produced in total by intelligent super computers will sky rocket the rate of discovery and cut 99% of people out of the work and eventually everyone out. And that’s not even assuming the machines get smarter than us which they will by orders of magnitude. Biology will resist AI takeover for longer than the physical sciences but not nearly by so long as people think. Unless you’re within 5 years of retirement the wave will hit you. Unless we stop further AI advances. And not to save our jobs but because this tech is uncontrollable and could kill us all.

Look at the kind of people who run the world and ask yourself are they really not gonna weaponize every breakthrough in AI? We’re not heading towards some utopia where everyone has high incomes and an abundance of whatever they want. That’s a dream. Wake up to reality before it becomes a nightmare.

It seems clear that people in every field are going to insist that their field is special and too hard for AI to take over. First the coders, now the mathematicians, next up theoretical physics, etc. At each step experts pontificate about how hard their field is and how different from those who already fell. History does rhyme I guess.

I’m amazed by how many people don’t know that science and math are really different fields. Yes we use math in science but the math we use is a small subset of math. Most pure mathematicians, no matter what they might claim, care about pure math, not so much its applications. Psychologically, basic math people and scientists are also radically different. Mathematicians love to solve puzzles for the enjoyment of doing so. They love elegant solutions to pithy mathematical riddles. Scientists in contrast are more utilitarian although we do have a passion for our questions.

We don’t understand the problem solving of AI in general terms yet but there are some findings that suggest that what’s happening is what common sense would tell you. When AI learns to add numbers it does not learn to statistically associate 2+5 with the answer 7. Nor does it learn by rote every possible case of addition. AI learns rules of addition which can be applied to novel cases of addition. The few other simple math cases we’re starting to understand are the same. It may not learn the same rules we use but it learns rules. There is also evidence that AI is learning informal logical rules as well in the context of language and other problem solving activities. In laymen’s terms, AI does seem to think. It’s not a statistical parrot, although it is also capable of that in some cases.

The tech of today was the science fiction of a couple generations ago. Now that we’re in the exponential phase of growth in tech with AI the lag time between sci-fi and real life is shrinking drastically. ‘That’s sci-fi’ or ‘you watch too many movies’ is no longer a rational criticism of most warnings about tech!

Many people don't know how AI works. It's complicated but I'll say a few words and build on it over time. An LLM is a type of neural network. A neural network is a type of mathematical system. How it works is simple in principle but complex in practice. You take your input, whether it be a picture or a question, and you turn it into a list of numbers, called a matrix (or vector) in the language of math. You then perform a long series of mathematical operations on this list of numbers until you get an output vector. This is a list of numbers we can translate into an answer like the picture was of Bob, or the answer to your question is "Tuesday." The mathematical operations in between the input and output lists of numbers is the neural network. That's basically what ChatGPT and Claude are. I'm painting in broad strokes, of course. What I've explained is called inference. This is when you use the finalized version of the LLM. Training is what we call finding the right settings that allow network to turn your input list into the right output list. That's enough for now. Its also enough to know why AI is not programmed, or code. Its math. There is a program, called a harness, that feeds queries and gets answers from the LLM, and filters them and so forth, but it doesn't problem solve. The math does that. And you can't program math; it is what it is. You can parameterize a mathematical system to try to get it to do what you want, of course, but for a highly complex system this is difficult, and sometimes impossible. But I'll save that for another tweet.

AI can hack almost anything, which is why everyone is so unprepared for what’s happening. They can’t prepare. Maybe this is actually good. If the fact that we can’t protect banks sinks in then maybe the fact that ultimately we won’t be able to protect ourselves will also sink in!

The key thing to keep in mind about AI is not what it can now do but rather what it was doing 2 years ago. Because this train is not stopping with respect to growth in ability. The low hanging fruit hasn’t even been plucked as far as alternative model designs go. The issue is compute. It’s expensive to grow these models so not a lot of experimentation can happen at the level of huge models. But once better hardware and more data centers come online there’s going to be an explosion of new approaches and many will work. We are now witnessing the birth of AI. It will explode in ability relatively soon. If we don’t stop it within a few years we likely won’t see the end game of a planet of silicon life with us a chapter of their history.

I think anyone who can’t separate the business issues of AI from the existential risk is some kind of capitalist caveman. The internet exploded despite the dot com crash and AI also does not need particular companies like OpenAI or Anthropic. It’s bigger than that.

AI is a threat to the survival of humanity. I intend to post on this subject to get the message out that this tech is far more dangerous than most people realize. Current AI is equivalent in some ways to the earliest Wright brothers airplanes. Primitive but with the obvious potential to explode in capacity. This explosion in ability will come without warning and once it happens it may be too late to do anything to stop it from wiping us out. So we have to do something now to stop further AI development.