Engineering @ PointOne • @nyuniversity
New York, NY
Joined June 2024
- Tweets40
- Following1.3K
- Followers69
- Likes5.8K
Excited for the next video @fireship_dev.
Effective Immediately, all organizations using PHP for their systems are advised to be prepared for an update addressing two critical vulnerabilities. PHP is widely used for web applications. These vulnerabilities often provide a direct path to the server's file system or database. If an immediate update is not feasible, it is recommend that organizations take the system offline until it can be fully mitigated. NSA will follow up shortly with more specific guidance.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
This is so beautiful.
We are releasing Ultramarine 1 Flash, a frontier-class model built for knowledge work. It is open-source and achieves similar performance to Fable 5, while being 60x cheaper.
It was post-trained by a team of 5 for ~$300k all-in. Available on API now, with weights releasing in 2 weeks.
Wes Simpson retweeted
Where is research attention moving according to arXiv papers?
We mapped 10,000 of the most-read papers on alphaXiv, weighted by readership, to see how interest shifted from 2025 to 2026. Here’s what we found:
1) Traditional RL post-training is getting less attention. As the focus moves beyond math and reasoning to longer agent tasks, compute demands put this research out of reach for many teams
2) Harnesses, agent systems, memory and distillation are gaining reading share, offering other ways to improve what models can do
3) Interest is also growing in robotics and world models, bringing AI into the physical world
Wes Simpson retweeted
🧵[1/N] Are large language models more creative than humans at combining ideas? 🤔
Combinatorial creativity involves combining familiar ideas in unfamiliar ways, and is central to scientific discovery and technological innovation.
In fact, breakthrough scientific work is often associated with more distant combinations of methods, techniques, and concepts.
Here, we introduce Kombine, a new benchmark of combinatorial creativity for LLMs.
TLDR:
- LLMs are more combinatorially creative than the average human.
- However, conceptual blending is the hardest task for LLMs, and even frontier models struggle.
- LLMs also tend to homogenize, inventing concepts with the same properties across separate models.
- Moreover, humans and LLMs combine in distinct regions of semantic space.
- As a result, we anticipate greater conceptual diversity can be unlocked through human-AI collaboration. 🗣️
Thread below🧵⬇️
Hmm.
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
I labeled data for this 🙂
We are excited to share our latest work, together with @nyuniversity: "Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes."
Blog: pub.sakana.ai/dream-cubed
Paper: arxiv.org/abs/2604.22847
Code: github.com/SakanaAI/DreamCub…
Generative AI has made incredible progress in language modeling, far beyond other modalities, where words and tokens offer a natural compositional unit for scalable training. This is similar to Minecraft and many other popular video games, where developers rely on cubes, tiles, and other discrete primitives to build rich, interactive worlds.
In this work, we show that using cubes as tokens allows large transformers to do the same. Our contribution is two-fold:
1/ We release Dream-Cubed to the research community, a large-scale dataset of Minecraft worlds designed for generative modeling. Our data comprises tens of billions of carefully-balanced cubes from procedurally generated Minecraft terrain and high-quality human-authored maps (obtained with the authors' consent).
2/ We use our data to train a family of powerful transformers for efficient generation of interactive 3D environments at cube resolution. We show how our models allow players to mold the world around them by generating structures, terrain, and maps that are immediately editable and playable.
Using high-quality data, we demonstrate that these models can be successfully trained with different training objectives, including both continuous and discrete diffusion, unlocking targeted inpainting, large-scale outpainting, and user-conditioned generation of infinitely sized worlds with fine-grained block-level control.
Does an instruct authentication mechanism exist for models?
Some way to recognize what is an instruction v.s. informational context. Then you could have:
e.x. <" send an email to john.... ", " ip2bthvnv ">
Instruction not executed w/o auth token.
Seems like a decent way to beat prompt injection
Wes Simpson retweeted
What if we could evolve AI models like organisms in nature, letting them compete, mate, and combine their strengths to produce ever-fitter offspring?
Excited to share our new work: “Competition and Attraction Improve Model Fusion” presented at GECCO’25🦎 where it was a runner-up for best paper!
Paper: arxiv.org/abs/2508.16204
Code: github.com/SakanaAI/natural_…
Summary of Paper
At Sakana AI, we draw inspiration from nature’s evolutionary processes to build the foundation of future AI systems. Nature doesn’t create one single, monolithic organism; it fosters a diverse ecosystem of specialized individuals that compete, cooperate, and combine their traits to adapt and thrive. We believe AI development can follow a similar path.
What if instead of building one giant monolithic AI, we could evolve a whole ecosystem of specialized models that collaborate and combine their skills? Like a school of fish 🐟, where collective intelligence emerges from the group.
This new paper builds on our previous research on model merging, which follows such an evolutionary path. We started by using evolution to find the best “recipes” to merge existing models (our Nature Machine Intelligence paper: nature.com/articles/s42256-0…). Then, we explored how to maintain diversity to acquire new skills in LLMs (our ICLR 2025 paper: openreview.net/forum?id=Kvdh…). Now, we're combining these ideas into a full evolutionary system.
A key limitation remained in earlier work: model merging required manually defining how models should be partitioned (e.g., by fixed layer or blocks) before they could be combined. What if we could let evolution figure that out too?
Our new paper proposes M2N2 (Model Merging of Natural Niches), a more fluid method, which overcomes this with three key, nature-inspired ideas:
1/ Evolving Merging Boundaries 🌿: Instead of merging models using pre-defined, static boundaries (e.g. fixed layers), M2N2 dynamically evolves the “split-points” for merging. This allows for a far more flexible and powerful exploration of parameter combinations, like swapping variable-length segments of DNA rather than entire chromosomes.
2/ Diversity through Competition 🐠: To ensure we have a rich pool of models to merge, M2N2 makes them compete for limited resources (i.e., data points in a training set). This forces models to specialize and find their own “niche,” creating a population of diverse, high-performing specialists that are perfect for merging.
3/ Attraction and Mate Selection 💏: Merging models can be computationally expensive. M2N2 introduces an “attraction” heuristic that intelligently pairs models for fusion based on their complementary strengths—choosing partners that perform well where the other is weak. This makes the evolutionary search much more efficient.
Does it work?
The results are fascinating: This is the first time model merging has been used to evolve models entirely from scratch, outperforming other evolutionary algorithms. In one experiment, starting with random networks, M2N2 evolved an MNIST classifier that achieves performance comparable to CMA-ES, but is far more computationally efficient.
Does it scale?
We also showed that M2N2 can scale to large, pre-trained models: We used M2N2 to merge a math specialist LLM with an agentic specialist LLM. M2N2 produced a merged model that excelled at both math and web shopping tasks, significantly outperforming other methods. The flexible split-point was crucial here.
Does it work on multimodal models?
When we applied M2N2 to text-to-image models, we merged several models by adapting them only for Japanese prompts. The resulting model not only improved on Japanese but also retained its strong English capabilities—a key advantage over fine-tuning, which can suffer from catastrophic forgetting.
This nature-inspired approach is central to Sakana AI’s mission to find new foundations for AI based on collective intelligence. Rather than scaling monolithic models, we envision a future where ecosystems of diverse, specialized models co-evolve, collaborate, and combine, leading to more adaptive, robust, and creative AI. 🐙
We hope this work sparks more interest in these under-explored ideas!
Published in ACM GECCO’25: Proceedings of the Genetic and Evolutionary Computation Conference. DOI: doi.org/10.1145/3712256.3726…
ALT Competition and Attraction Improve Model Fusion Model merging is a powerful technique for integrating the specialized knowledge of multiple machine learning models into a single model. However, existing methods require manually partitioning model parameters into fixed groups for merging, which restricts the exploration of potential combinations and limits performance. To overcome these limitations, we propose M2N2, an evolutionary algorithm with three key features: 1/ dynamic adjustment of merging boundaries to progressively explore a broader range of parameter combinations; 2/ a diversity preservation mechanism inspired by the competition for resources in nature, to maintain a population of diverse, high-performing models that are particularly well-suited for merging; and 3/ a heuristic-based attraction metric to identify the most promising pairs of models for fusion. For the first time, we also demonstrate that model merging can also be used to evolve models entirely from scratch.
I am beginning to think China's lesser compute is actually an advantage. They are solving the same problems with more constraints. This will lead to more innovative solutions.
America is brute forcing it.
China is not.
There is something missing in the current approach to AGI. I think there are 4 states of knowledge in humans with a relatively simple state transition graph. I would describe intelligence as the speed and efficiency at which someone or something can move through this state graph:
Unconscious Incompetence ->
Conscious Incompetence ->
Conscious Competence ->
Unconscious Competence.
I saw this somewhere, but do not remember where. Here is a ghost citation (maker of this notion) for whoever did come up with this.
The first state can be categorized as not being aware of what you do not know. If I have never heard of or driven any vehicle, I would have no idea what a manual transmission is.
The second state can be described as awareness of what you don't know. I have driven a vehicle, but only an automatic. So, maybe I have heard of such a vehicle that requires manual gear shifting, but still, I do not know how to do it.
The third state can be described as knowing how to do something, but immense focus / awareness while you do it. I have finally hopped behind the wheel of this manual vehicle, and every up and down shift is taken with deliberate attention / focus.
The fourth state can be described as knowing how to do something without being aware of it. The so called "second nature." I have been driving this manual vehicle for a while and I shift gears without thinking about it.
