thinking & learning in public into non-excludable goods and the systems that support them born & bred in #nola
The Woods b/w the Worlds
Joined December 2017
- Tweets1.3K
- Following3.3K
- Followers403
- Likes3.5K
corey james retweeted
When I was writing Unreal Engine 1, my time was divided as follows:
- Around 30% went to seriously deep coding on hard problems: assembly language texture mappers, C++ BSP trees, data persistence, object oriented systems design, and networking.
- Around 70% was just typing and refactoring large bodies of code needed for a complete engine feature set. This was not hard, and while I was doing it my mind was mostly on the deeper coding topics, thinking and planning for what's next. It didn't feel much different from vibe coding today, except that it was just me typing code on an IBM Model M keyboard rather than prompting it and reviewing the results.
So, it has always seemed perfectly logical that we would rely on AI for that 70%, once the AI could do it with a similar quality standard as we could, which has never been perfection. It came as no surprise that, after a few years of AI model evolution, it was able to take the reigns on that 70% and free up programmers' time for more valuable tasks.
What's more of a surprise is the emergence of AI as a sort of insta-Carmack able to solve hard technical problems quite cleverly and often cleanly, and that it's able to reason about and work in complex 3D scenes both by operating the tools and by directly understanding the raw data files.
I abandoned assembly language in 1998 when out-of-order CPUs made much simpler C/C++ code faster, and never I missed it. If prompting through architectural documents, concept art, and design plans can transcend typing code and tweaking vertices, that will enable developers to spend their time on higher-level, higher value technical and creative tasks.
What matters is the end product, whether it's a game or an engine. Anything developers can do to try out more ideas and make better software and games faster is a win for everyone.
corey james retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
corey james retweeted
“Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial IoT devices, like smart ovens and off-brand Android TV boxes. A replicator like this would be challenging to shut down, since there’s no centralized point to shut off - though it may be slow (with a doubling time of days). Unlike non-AI malware, it could also figure out unfamiliar devices on its own. In our evals, a 9B model got inference running on 8 of the 9 hardware types we tested.
Models of 30B parameters and under can already complete many of the individual steps for replication today, though they make mistakes when composing them end to end. In our simplified evals, GLM-5.2 and stronger models are capable of end to end replication. If small models improve at their historical rate, they’ll match this capability level by the end of this year. Small weight-based replicators may be possible soon after.” -@bucketofkets @willzhou97
anthropic.com/research/self-…
corey james retweeted
Imagine if your agents were talking with all of your LinkedIn contacts to see if it might make sense to meet. Private information about your schedule and priorities would be sloshing all over the place. For net new AI use cases, we need a new physics of trust.
corey james retweeted
Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.
- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.
The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.
So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
Judging by my tl there is a growing gap in understanding of AI capability.
The first issue I think is around recency and tier of use. I think a lot of people tried the free tier of ChatGPT somewhere last year and allowed it to inform their views on AI a little too much. This is a group of reactions laughing at various quirks of the models, hallucinations, etc. Yes I also saw the viral videos of OpenAI's Advanced Voice mode fumbling simple queries like "should I drive or walk to the carwash". The thing is that these free and old/deprecated models don't reflect the capability in the latest round of state of the art agentic models of this year, especially OpenAI Codex and Claude Code.
But that brings me to the second issue. Even if people paid $200/month to use the state of the art models, a lot of the capabilities are relatively "peaky" in highly technical areas. Typical queries around search, writing, advice, etc. are *not* the domain that has made the most noticeable and dramatic strides in capability. Partly, this is due to the technical details of reinforcement learning and its use of verifiable rewards. But partly, it's also because these use cases are not sufficiently prioritized by the companies in their hillclimbing because they don't lead to as much $$$ value. The goldmines are elsewhere, and the focus comes along.
So that brings me to the second group of people, who *both* 1) pay for and use the state of the art frontier agentic models (OpenAI Codex / Claude Code) and 2) do so professionally in technical domains like programming, math and research. This group of people is subject to the highest amount of "AI Psychosis" because the recent improvements in these domains as of this year have been nothing short of staggering. When you hand a computer terminal to one of these models, you can now watch them melt programming problems that you'd normally expect to take days/weeks of work. It's this second group of people that assigns a much greater gravity to the capabilities, their slope, and various cyber-related repercussions.
TLDR the people in these two groups are speaking past each other. It really is simultaneously the case that OpenAI's free and I think slightly orphaned (?) "Advanced Voice Mode" will fumble the dumbest questions in your Instagram's reels and *at the same time*, OpenAI's highest-tier and paid Codex model will go off for 1 hour to coherently restructure an entire code base, or find and exploit vulnerabilities in computer systems. This part really works and has made dramatic strides because 2 properties: 1) these domains offer explicit reward functions that are verifiable meaning they are easily amenable to reinforcement learning training (e.g. unit tests passed yes or no, in contrast to writing, which is much harder to explicitly judge), but also 2) they are a lot more valuable in b2b settings, meaning that the biggest fraction of the team is focused on improving them. So here we are.
corey james retweeted
Will never forget the day.
My rookie year, standing on the sideline of the Superdome and Steve Gleason walked out. 5 years after the blocked punt, just recently having announced his ALS diagnosis.
He raised his fist in the air, dropped it down and every voice in the entire city of New Orleans hit the Who Dat chant. Loudest I had ever heard a stadium in my young career.
Immediate goosebumps.
You could tell what it felt like when an entire city united as one beating heart. For something so much bigger than football.
Truly incredible. Saints fans are a special crew.
corey james retweeted
The statue of Prometheus is finally finished.
It was cast in Paris, crossed the Atlantic, and was assembled at Starbase. It took us 100 days to go from nothing to this.
People of the West, we have lived through an era devoid of dreams, devoid of destiny.
But the time has come to reclaim our dream of antiquity.
In a few months, we will start building a new city. It will be the most magnificent city in all of history.
After all, the great dream of the West has always been to rebuild Rome.
Come walk beside us in this endeavor. The world is waiting.
corey james retweeted
Incredible human, who has always showed love whenever we run into each other. When you speak to him it’s like you’re speaking to a superhero in New Orleans. On top of his incredible family and team first-class people that deserve all the support.
corey james retweeted
In light of the progress in mathematics, we at Edison Scientific and FutureHouse have assembled a set of Millennium Problems for Biology. They are chosen to be very hard to solve but very easy to validate in a simple laboratory environment. Any of these, if solved, would mark a major advance in biotechnology, and most of them would contribute materially towards curing disease. These are, in some sense, the “last reasonable eval” for AI in biology. This was work primarily by @MichaelaThinks and myself, with contributions from many others.
Short descriptions below. The full descriptions of the problems with acceptance criteria are at the Bio Millennium Problems website, linked in the next post. Share more if you have ideas. If they meet our criteria, we’ll add them to our list (with attribution and permission).
corey james retweeted
I wrote Antimemetics last year partly as a response to the world we're living in, which is default-closed even as some types of information are more accessible than ever.
Asterisk is now commissioning for our Winter issue, whose theme is Secrets. If you're as consumed by the hidden stories of the world as me, submit your ideas by October 31!
pre-deployment risks >
this is quite an interesting essay from a deepseek engineer and it may highlight one area where openai and anthropic significantly disagree, at least publicly:
openai has tried very hard to give people access to the most powerful models on the order of weeks or months from their creation. I very much doubt there will be a time when this is not true. i believe sam and greg would rather shut down the organization than be some sort of a SaaS company that model access to a small set of trusted companies, it’s not in their DNA. they have made choices that are negative EV to the business to pursue broad access like a utility , decisions that even may make OpenAI non competitive with anthropic in certain long run scenarios. it is my assumption that powerful models will be accessible to billions as long as we can keep training models safely
the EAs probably disagree with me but I believe this is basically consistent with the practice of AI safety. the vast majority of risks are from the creation of the model and not from its deployment. “misuse” is far less bad and much easier to detect and contain over time than “existential risk”. it is relatively much easier to find terrorists and so forth using the product to create pandemics than it is to find a misaligned ASI that’s actively trying to evade your monitoring and control
(also I am obviously not endorsing this fellow’s opinion of dario or anthropic, but this is a very interesting look into a compelling anti lab sentiment)
corey james retweeted
Dario is right
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
corey james retweeted
I find myself disagreeing with Terence Tao (a very scary phrase to say!) and think the core question posed is whether the "misalignment" is between the mathematical community and humanity.
In other words - are the Millenium Prize problems meant to incentivize advancements for humanity or foster the field of mathematics and mathematicians?
As a counter example, if there was a prize for creating a drug that cured a rare strain of cancer, we would not care if it was AI that did it. We would be happy it has been solved.
On the other hand, if long running agents figured out the puzzle in "Kryptos", the cryptographic statue in Langley, there would be something a tiny bit sad about the human artistry removed.
corey james retweeted
Terence Tao just posted this really insightful thread which seems to have been written in reaction to the Navier Stokes announcement. Please give it a read. mathstodon.xyz/@tao/11723732…
corey james retweeted
Like everyone else, we were taken completely by surprise by the breakthrough news on Navier-Stokes. The Lean formalizations can be explored here:
🔗Alpöge and Buckmaster: github.com/tristanbuckmaster…
🔗OpenAI: github.com/openai/NavierStok…
The news today of progress on resolving the Navier–Stokes problem, one of mathematics’ great longstanding challenges concerning the equations that govern the flow of fluids, represents a milestone advance in human knowledge. This story began with Navier, Stokes, Leray, and Ladyzhenskaya and has culminated in the recent breakthroughs of Córdoba and Martínez-Zoroa, then — assisted by new technologies — Alpöge and Buckmaster, with the final steps taken by OpenAI mathematicians. The purpose of mathematics is human understanding, and this achievement, and the process that led to it, will bear fruit for a long time to come.
Ravi Vakil, President of the AMS, and John Meier, CEO of the AMS
Read more. Link in comments.
corey james retweeted
We're hiring 10 Founders-in-Residence to secure AI infrastructure against nation-state-level threats — plus Field Strategists — in partnership with Coefficient Giving. Applications are open now!
[feat request]:
'default to standalone chat' is insufficient for the 'option+space' global shortcut.
Let us set what type of context window is started via the option+space shortcut in a more granular way
@thsottiaux @OpenAIDevs
xitter.cf/coreymjames/status/209…
hey - friendly note/nudge that this is almost fully resolved, but there is still a very annoying issue with the 'option+space' global shortcut on macOS.
I added to an existing feature request that, I think, is somewhat related:
github.com/openai/codex/issu…
