John Spindler retweeted
Delighted to share a landmark preprint from our lab, using genome language models and AI agents for biological discovery and genome mining!
The stretches of DNA between protein-coding genes are often called the genome’s “dark matter,” but they are far from empty. They encode RNA molecules that sense metabolites, regulate genes, and control some of molecular biology's most important tools (such as programmable genome editors e.g. CRISPR/bridge)
Much of this biology is unexplored. One reason is that RNAs can retain similar structures and functions even as their DNA sequences change dramatically, making their relationships difficult to recognize with conventional computational tools
Massive congratulations ✊✊🎉🚀🍾
We’ve raised a $45m Series A, co-led by
@foundersfund and The @OpenAI Startup Fund, to build cancer therapy that learns. Technology is rapidly changing medicine, but data loops and rapid adjustments are still largely missing. When you start cancer treatment, knowing whether what you’re doing is working is too slow. Clearly there is a better way. Build systems to watch cancer in real time, then feed that data into a system that itself can treat the disease. This seeds a future where, unlike drugs, we develop medicine that continuously learns to improve itself. We've been building neurotechnology and BCI platforms to do just that.
@coherenceneuro
John Spindler retweeted
We’ve raised a $45m Series A, co-led by
@foundersfund and The @OpenAI Startup Fund, to build cancer therapy that learns. Technology is rapidly changing medicine, but data loops and rapid adjustments are still largely missing. When you start cancer treatment, knowing whether what you’re doing is working is too slow. Clearly there is a better way. Build systems to watch cancer in real time, then feed that data into a system that itself can treat the disease. This seeds a future where, unlike drugs, we develop medicine that continuously learns to improve itself. We've been building neurotechnology and BCI platforms to do just that.
@coherenceneuro
Massive congrats to our portfolio startup @Recursive_SI team for the launch of Flint. Running a coding model fast on a laptop, with the code never leaving the machine, is impressive. A great example of RSI working in the field. Much more to come from this amazing team.
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
John Spindler retweeted
This is actually more a side-effect of the death of pretraining.
As transformers saturate, most pretraining results are tweaking data recipes, not massive stepwise changes in performance.
Turns focus to innovation in postraining and test time compute. This is good for competition, since pretraining concentrated resources in a few labs.
GLM 5.3’s biggest leap came from post-training alone.
“GLM 5.3 was built entirely by post-training the previous model. For me, that was a wake-up call.”
“Post-training is far more GPU-efficient than pre-training from scratch. Plenty of companies can’t afford to train a model from scratch, but they have proprietary data. They can post-train themselves and beat some of the closed models.”
“There’s still so much juice left to squeeze from post-training. We compared GLM 5.3’s cyber capabilities with 5.2, and it’s a night and day difference. That’s all just post-training.”
John Spindler retweeted
The outpouring of interest in our paper has been … a bit insane.
May I make four points?
1. We have not shown that “consciousness has a frame rate.”
I can see why people draw this conclusion: we report that inferotemporal and prefrontal cortex alternate between conscious and suppressed states roughly every 60 ms during binocular rivalry.
Consciousness depends on many brain areas, and our paper reports results from only a few select nodes.
We still don’t know how conscious access works--the actual recipe for generating conscious experience. What we have shown is that deep in the brain there is an extremely strong signal, rich enough to reconstruct a high-dimensional face in detail, that is inaccessible to consciousness. As I argued in the original tweet, this makes the problem of conscious access a lot more experimentally tractable. But to claim “consciousness has a frame rate,” one would need to have actually solved the mechanism for conscious access and shown that this mechanism operates at a frame rate.
While we currently don’t know conclusively one way or the other (see VanRullen & Koch, 2003 for a review of positive evidence), I personally am skeptical that consciousness has a frame rate. I believe the brain, within posterior parietal cortex, has continuous tracking mechanisms that actively represent the persistence of matter, including ourselves as physical bodies. The coupling between these persistent tracking processes and feature representations across the brain (like those in face patches) is, I think, the essential mechanism of conscious access.
This coupling need not happen at the rate of inferotemporal alternations (and inferotemporal cortex itself may not even alternate outside of the artificial situation of binocular rivalry). As an analogy: imagine the area crucial for conscious access holding a steady drone note, while inferotemporal cortex alternates between two notes, one in harmony and one dissonant. Inferotemporal content might couple to the drone only at moments of harmony, yet experience would still be continuous, because the drone itself (the tracking process that I believe is at the core of conscious perception) never stops. And areas without conflicting content, like auditory cortex during visual rivalry, could stay in continuous harmony with the drone.
In sum, there is no reason to conclude from our study that consciousness is discrete, with a "frame rate."
2. Our results resemble Jezek et al. 2011 (as @jjsakon notes).
Jezek et al. trained rats in two boxes that were identical except for their lighting cues, so that hippocampal place cells formed two distinct maps, one per box. They then “teleported” the rat by abruptly switching the lights. For a few seconds afterwards, CA3 place-cell activity flickered between the two maps, synced to the theta cycle (~125 ms).
@StanDehaene first pointed this paper out to me, and reading it blew me away. To my mind it is the strongest evidence for a bistable attractor in the brain, because the competing maps are stored within the hippocampus itself.
I think the phenomenon we see may be very similar at the circuit level: a bistable attractor network within inferotemporal cortex that alternately converges to one of two patterns, as cells in face patches work together to build a coherent representation of the input. In our case, both images are present in the input, so the competition could arise within IT or be inherited from earlier areas such as V1--recording there is an important next step. (It also still puzzles me that we didn’t see strong coherence between face patches ML and AM.)
More broadly, what’s different about our paper is the brain area (inferotemporal cortex, often modeled as a feedforward DNN) and the ability to relate the two competing patterns to consciousness, with one granted conscious access and the other not.
3. Please comment on bioRxiv!
The Twitter attention has been cool, but I really hope colleagues will be generous enough to post detailed commentary on bioRxiv, where there are currently 0 comments. As @seemaychou points out, peer review doesn’t have to come from just three anonymous reviewers. We would be truly grateful for your critical feedback.
4. Stay tuned
This work was done at Caltech, Berkeley, in collaboration with @EngelTatiana at Princeton, funded by NIH and HHMI.
I hope to share our vision soon for how we can solve consciousness @AsteraInstitute
1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness.
New preprint: biorxiv.org/content/10.64898… (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)
John Spindler retweeted
1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness.
New preprint: biorxiv.org/content/10.64898… (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)
John Spindler retweeted
CLAUDE FOUND THE SEEDS
anthropic.com/research/ecdsa…
John Spindler retweeted
@GoogleDeepMind Gemini Embedding 2 is the SOTA embedding model for vector-based retrieval.
We benchmarked PageIndex against vector-based retrieval powered by Gemini Embedding 2, comparing both accuracy and cost on financial reports.
Run the benchmark: github.com/VectifyAI/PageInd…
John Spindler retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
John Spindler retweeted
Excited to launch Gemini 4 Argon, designed to help with hardest coding and knowledge work tasks with its frontier capabilities..
For developers and enterprises, it’s available in the paid tier of the Gemini API and the Gemini platform, for Ultra subscribers in Google Antigravity, and Ultra and Pro subscribers in the Gemini app and Search.
Read more: blog.google/innovation-and-a…
John Spindler retweeted
Great to be on MLST! My favourite part of the conversation was about the spaghetti code our RSI loop produced.
It ran experiments on 100 inner loop variations in eight days, probably close to what we ran by hand in two years, and the harness it found beat ours on held-out benchmarks.
We still prefer our own code because it’s much more understandable. But we don’t know whether that preference reflects a human bias, since the generated code performs better empirically.
> An AI rewrote its own coding agent for 8 days.
> @zhengyaojiang post calling it the first evidence of recursive self-improvement got 1.8M views here.
And no, he doesn't expect a singularity!
John Spindler retweeted
Announcing PageIndex SDK v0.4.0.
We've optimized tree generation, and for the first time:
Tree indexing is now cheaper than vector indexing.
PageIndex, powered by GPT-6 Luna, achieved ~15% lower indexing costs than vector indexing with Gemini Embedding 2 in our 1,000-page financial report benchmark.
Better retrieval. Lower indexing cost.
→ Explore the cookbook:
github.com/VectifyAI/PageInd…
John Spindler retweeted
Just as mathematicians are now learning how many insights lie in the "convex hull" of existing ideas, the rest of science is about to learn this lesson at even more dramatic scale. Few realize the ridiculous inefficiency of the pipeline from theoretical innovation to practical use.
Early versions of GPT made it easier to find existing knowledge. But finding a theorem is not the same as recognizing that it offers a radically different way to conceptualize a problem. Doing the latter requires expertise across fields that almost nobody possesses.
Most of the people doing applied work are completely unaware of the conceptual tools that are being developed in more theoretical fields, and the people doing high-level theory do not understand the practical problems applied researchers are trying to solve. Almost no one doing practical work of any importance has an operational understanding of the web of ideas underlying the langlands conjecture or condensed mathematics, and virtually no one doing biology, chemistry or economics is knows anything about 90% of the math used in theoretical physics.
And for good reason! Most of these ideas would be completely useless. But the scope for possible innovation from drawing the rare useful connections is immense.
Think of Paul Samuelson borrowing ideas from thermodynamics to model equilibrium in economics, or Ed Witten using quantum field theory to revolutionize geometry. A few startling successes, not feasible to most: trying to venture into a more theoretical or more practical discipline to find insights of use is like searching for a needle in a haystack.
What I suspect we will see in the next few years is a scientific revolution brought on by models that combine deep theoretical insight with deep practical knowledge and judgment. Existing publicly available models are lacking in all three departments, but getting better. When searching for useful connections between distant fields becomes cheap, even a low hit rate could produce an explosion of innovation. Even without new ideas (which will also be forthcoming...), the convex hull of existing science would push far beyond the status quo.
John Spindler retweeted
Inductive reasoning has always been a bottleneck for LLMs thus autonomous research.
Glad there’s a benchmark for this now. Very curious on how fast the progress would be on this given building a RL environment seems to be non trivial
AI scientist agents are great at optimizing and finding solutions through endless trial and error. But is that really what science is about?
As Terrance Tao eloquently put, the role of math and science should be more than that – they are "lighthouses" that guide and inspire exploration through understanding and insights. Can AI agents discover novel insights through experimentation?
To study these gaps, we worked with domain experts to introduce EurekaBench, a benchmark spanning 6 science domains that evaluates an agent’s ability to discover genuinely new scientific insights.
(1/n)
John Spindler retweeted
12 months of progress on the jagged frontier:
1) AI got spikier, not rounder.
2) Maths, fluid reasoning, expert work: past human level.
3) There is almost no middle ground. Once a capability crosses human level, it often quickly becomes 100x+ cheaper than a human.
4) Cost keeps decreasing after capability increases stop. Language and knowledge look saturated but price keeps falling.
5) The gaps are not a budget issue. Human-level memory, self-monitoring and long-run planning are not for sale at any price.
AI is now a superhuman specialist that is sub-human at running itself.
John Spindler retweeted
From the essay:
Reading all those recent beautiful essays by Terry Tao, Kevin Buzzard and @wtgowers, I could not resist, but write up my deeply personal perspective on how I perceive and see things:
docs.google.com/document/d/e…
John Spindler retweeted
Sadly true. It gets especially expensive for self-improvement with frontier models, where you need to run trials, learn from them, and iterate repeatedly.
Prediction: universities will get priced out of AI evals work
We just spent $50k yesterday benchmarking 40 long horizon coding tasks
As we design harder tasks for agents, trajectories are getting longer and need more tokens. That means substantially more inference and compute. How is this economically feasible for academics working off university grants?
