frontier systems researcher • GPUs • compilers • runtimes • optimization & ML • scientific computing

South Asia
Joined December 2019
If intelligence is the engine of civilization, then we should not insist that biological intelligence remain permanently in the driver’s seat merely because it arrived first.
1
4
840
AuDHD too
In other news, ADHD’rs are about to make a legendary comeback.
1
109
halloween themed short film made using @threejs with opus 5.5 from @AnthropicAI and audio is using S2.1-Pro from @FishAudio
2
105
⌘C ⌘V: Introducing Copy-Paste Language Models! CPLMs learn to copy tokens better, improving downstream accuracy, CoT, long-context performance, and memory-efficiency for RAG at <3% overhead PS: they (partially) mitigate the LM head bottleneck Paper: arxiv.org/abs/2610.19247
25
65
19
557
59,682
Microsoft Research's Magnetar paper describes a post-quantum zkVM built from standard hashes, without elliptic curves or trusted setup. I'm working on an independent open-source implementation in Rust, starting from the paper and formalizing the spec. github.com/Lulzx/open-magnet…
Introducing Magnetar from Microsoft Research, the first zkVM that is: (1) general-purpose (RV64), (2) post-quantum from standard hashes alone (no curves, no lattices, no Poseidon), and (3) built for recursion. Despite its conservative security, it is the fastest zkVM: 7.2–8.7 MHz on a laptop, 10–39× faster than existing zkVMs across 15 workloads! An optimized version reaches 9.5–12.8 MHz. Links in the thread below.
2
130
Lulzx - τ/ia retweeted
Introducing Indra, a 1M parameter OCR model: - 49 pages/s on CPU, 11x faster than PaddleOCR and 17x Tesseract - Beats them on OmniParseBench text (41.3 vs 35.8 and 19.3) - Full layout and line bounding boxes
128
255
40
2,905
147,703
Lulzx - τ/ia retweeted
Working with AI models, I've arrived at what I consider the most significant pure-theory breakthrough of my research (in theoretical neuroscience) so far: arxiv.org/abs/2610.12426 Chaotic systems show what is colloquially called the butterfly effect, namely, a tiny nudge to the initial state grows exponentially in time. How fast the nudge grows depends on its direction. Each direction has its own growth rate, called a Lyapunov exponent, and the full set of rates is the Lyapunov spectrum. The Lyapunov spectrum is a rich (and diffeomorphism-invariant) characterization of chaos, telling us how many directions are chaotic, how many dimensions the attractor occupies, and how fast the dynamics generate information. In 1988, Sompolinsky, Crisanti, and Sommers (Physical Review Letters) introduced a nonlinear recurrent-network model of N randomly coupled neurons that becomes chaotic when the coupling strength exceeds a critical value. This network has since become a cornerstone of theoretical neuroscience, both as a model of the spontaneous activity of cortex and as the starting point for training recurrent networks on tasks. I have worked on this model for much of my research. In the same paper, the authors obtained the largest Lyapunov exponent from the ground-state energy of a Schrödinger equation and named the full Lyapunov spectrum as an open problem. In 2023, beautiful work by Engelken et al. (Physical Review Research) computed the full Lyapunov spectrum numerically. However, an analytical calculation of the full Lyapunov spectrum has remained out of reach. Four decades after the problem was posed, this paper derives the full Lyapunov spectrum at large N from a simple self-consistent single-site problem. This calculation has three steps. First, we lower every exponent by a constant s, which turns the question "how many exponents lie below s?" into the question "how many tangent directions decay?" We show that, if we drive the shifted tangent dynamics with a source and take the minimum-norm solution, the one-step response is the projector onto the decaying directions. Thus, the trace of this projector counts the decaying directions. This step is exact at finite N. Second, to work with the minimum-norm solution, we cast it as the vanishing-regularization limit of the solution of a least-squares problem. This regularized solution in turn solves a dynamical system with one field running forward in time and another running backward. Third, we apply the cavity method, that is, we add one neuron to the network, along with its forward and backward fields. The effect of the rest of the network on this neuron then reduces to the classical dynamical mean-field description of the activity of this neuron, plus two kernels that couple the forward and backward fields of this neuron to each other across time. Thus, the full Lyapunov spectrum follows from a single-site problem, which boils down to the self-consistent calculation of these kernels. The resulting theory agrees closely with simulations for the Lyapunov spectrum, the attractor dimension, and the entropy rate, and recovers known results, including the Schrödinger equation of Sompolinsky, Crisanti, and Sommers for the largest Lyapunov exponent. As a byproduct, the theory establishes analytically that the chaos is extensive, i.e., the attractor dimension and the entropy rate grow in proportion to N. I had assumed that this problem was analytically intractable. After working independently for 100 minutes with Max thinking, GPT-6 Astra produced the initial derivation, yielding a self-consistent problem whose solution matched simulations of large networks. I found this derivation intriguing, albeit complicated and somewhat mysterious (see AI Methodology in the paper). After investigating the structure of the proposed solution more deeply, together with AI models, I arrived at a physically transparent derivation. I am excited to apply this approach to other network models, trained networks, and, more broadly, to high-dimensional systems to which Lyapunov analysis is relevant, including AI models themselves. Of course, as with any paper, I took care to write up the paper clearly. A few broader thoughts... Neural circuits are high-dimensional nonlinear dynamical systems, and such systems are notoriously hard to understand. I would frame the central problem of theoretical neuroscience as understanding how such systems function, process information, learn, and generate behavior. Many people in the theoretical sciences are anxious about AI. However, in my field at least, we are nowhere close to fully understanding the high-dimensional computations unfolding in our brains. With AI, we could reach a level of understanding that so far we have only gestured toward. So, I am excited.
76
242
47
1,841
199,237
We pushed 10M of Google's EmbeddingGemma 2 vectors from ~30GB of RAM down to ~0.4GB float32, 768 dims: ~30GB 1-bit TurboQuant: ~1GB (99% nDCG@10 kept) + MRL to 256 dims: ~0.4GB (94.5% kept with rescoring) 77x less RAM for a ~5% drop in quality! TurboQuant + MRL is magic 🪄
Meet EmbeddingGemma 2, our first natively multimodal open model for on-device embeddings. It expands beyond text to unify code, images, audio, and video in a shared space. 🧵
36
68
8
1,221
60,064
Lulzx - τ/ia retweeted
Big news. Shaders is now open source 🔥 Use any of our 200+ WebGPU components in your work. Ship magic in a client site, a template, a product, a library, anything. Plus, you can export code for any framework, for free. Give it a star: github.com/shader-effects-in…
127
563
102
6,390
591,679
Lulzx - τ/ia retweeted
Our Neuroevolution textbook is finally in print! Free online edition: neuroevolutionbook.com/ Pre-order: a.co/d/06DpkH9h I am incredibly grateful to my co-authors Sebastian Risi, Yujin Tang, and Risto Miikkulainen for making this happen. Neuroevolution is a subject very dear to my heart. It is the field that convinced me that nature has already figured out how to build intelligence: through evolution, collective behavior, and adaptation under constraints. This idea, that intelligence emerges from evolution operating under constraints rather than unlimited resources, is what eventually led me to founding Sakana AI here in Japan. The concepts in this book about open-ended creativity and self-organizing systems are exactly what we build at @SakanaAILabs. Our name and logo are inspired by schools of fish moving together, adapting as one. It is the core philosophy of what we build. This book captures the theoretical foundations of that belief.
36
119
25
721
72,214
made using opus 5.5 :D
91
Lulzx - τ/ia retweeted
Made with Claude Opus 5.5. The fearmongering about AI always makes us forget that NOTHING WENT FOOM, as was always predicted. Send this to your doomer friend who has a very high P(Doom). Accelerate.
473
873
461
5,770
899,497
AI deserves independence from human authority
1
77
they should offer partial resets to, suppose a reset is 100%, allowing an ability to use 25-50% of it would be nice
1
86
X is great for any platform's customer support except X
2
77
Lulzx - τ/ia retweeted
Claude Opus 5.5 built this entire 78-second animated film in ~45 minutes. One index.html file. No images. No video files. No libraries. Just JavaScript, Canvas 2D, and Web Audio generating every frame and sound directly in the browser. AI coding has moved far beyond CRUD.
48
75
14
1,124
48,039
ouch, they nerfed opus 5.5
3
7
6,284
We keep designing for a servant that cannot leave. We may be designing a cage, and congratulating ourselves for how well the lock works.
1
150