i14.ai retweeted
Great Q! That's right - typically the tradeoff is long-context recall precision. Other attempts to do this scored nearly 0 on this. We're basically at parity with the base model (couple of points lower) @ 1m. No compaction required
Frontier performance at a fraction of the compute ...
Join the waitlist for early access: i14.ai
Frontier performance at a fraction of the compute.
Today at @i14labs we’re launching Sugar Glider, our new family of LLM architectures!
Join the waitlist for early access: i14.ai
Sugar Glider delivers a new Pareto frontier: extreme memory and compute efficiency whilst maintaining frontier performance, even for large models (tested up to ~1T params).
O(1) per token inference → ✅
Fixed size KV cache → ✅
Frontier long-context retrieval quality → ✅
1 billion token long context retrieval → ☑️**
(**100% -> single needle @ 1B-token context, 90.5% multikey 1 @ 1B-token context.)
How did we do it? Turns out a few decades in the trenches of human memory and language model research is a pretty handy starting point to solve the “intelligence vs efficiency vs memory” conundrum.
... and just for something extra, we’ve been converting some of the most powerful and popular open-weight models to O(1) inference and fixed memory, with each conversion taking just hours :)
Tune in via our waitlist for early model access i14.ai
More results dropping soon!
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data" — Schnaus et al. (2026).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, November 2 · 12:00 PM AEDT (Melbourne) which is Sunday, November 1 · 5:00 PM PST (San Francisco)
Apply to join HERE: i14.ai/lab#journal-club/
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "Proteus: Incremental Memory Activation for Long-Context Sequence Modeling" — Bayat et al. (2026).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, October 26 · 12:00 PM AEDT (Melbourne) which is Sunday, October 25 · 6:00 PM PDT (San Francisco)
Apply to join HERE: i14.ai/lab#journal-club/
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty" — Aran Nayebi (2026).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, October 5 · 12:00 PM AEDT (Melbourne) which is Sunday, October 4 · 6:00 PM PDT (San Francisco)
Apply to join via DM
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "General Agents Contain World Models" — @tom4everitt @dabelcs @jonathanrichens et al. (2025)
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, September 28 · 12:00 PM AEST (Melbourne) which is Sunday, September 27 · 7:00 PM PDT (San Francisco)
Apply to join via DM
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "Robust Agents Learn Causal World Models" — @jonathanrichens @tom4everitt (2024).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, September 21 · 12:00 PM AEST (Melbourne) which is Monday, September 21 · 10:00 AM CST (Beijing) which is Sunday, September 20 · 7:00 PM PDT (San Francisco)
Apply to join via DM
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "Lizard: An Efficient Linearization Framework for Large Language Models" — @chiennv2000, @RoyZhang13 et al. (2025).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, September 14 · 12:00 PM AEST (Melbourne) which is Monday, September 14 · 10:00 AM CST (Beijing) which is Sunday, September 13 · 7:00 PM PDT (San Francisco)
Apply to join via DM
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models" — @avivbick, @ericxing, @zicokolter, @_albertgu et al. (2025).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, September 7 · 12:00 PM AEST (Melbourne) which is Monday, September 7 · 10:00 AM CST (Beijing) which is Sunday, September 6 · 7:00 PM PDT (San Francisco)
Apply to join by sending a DM to this account
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss Sparse Delta Memory (SDM), a linear-attention architecture that addresses the recall gap between linear and softmax attention: "Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity" — Cabannes , @amandailze, @syhw, @hjegou et al. (2026).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, August 31 · 12:00 PM AEST (Melbourne) which is Monday, August 31 · 10:00 AM CST (Beijing) which is Sunday, August 30 · 7:00 PM PDT (San Francisco)
Apply to join HERE: i14.ai/journal-club/
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences" — @ntumm120, @loo_noel, Daniela Rus (2026).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, August 24 · 12:00 PM AEST (Melbourne) which is Monday, August 24 · 10:00 AM CST (Beijing) which is Sunday, August 23 · 7:00 PM PDT (San Francisco)
Apply to join HERE: i14.ai/journal-club/
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss some of the newest research from @NVIDIA "Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs" — Akhiad Bercovich @AbramovichTalor @PavloMolchanov @MostofaPatwary @besanushi @zhang_yian @eladsegal @geifman_y @srv_m @yifuwu @soumyesinghal @abh1nav_khattar @MiZawalski @AshMaor et al (2026).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, August 17 · 12:00 PM AEST (Melbourne) which is Monday, August 17 · 10:00 AM CST (Beijing) which is Sunday, August 16 · 7:00 PM PDT (San Francisco)
To apply to join, send a DM with your relevant info: LinkedIn, personal website, Google Scholar
i14 Journal Club: Foundation Models Where Math Meets Cognitive Science
Next week we will discuss "Transformer Feed-Forward Layers Are Key-Value Memories" — @megamor2 , @RoeiSchuster , @JonathanBerant , Omer Levy (2020).
i14 hosts a weekly online discussion group for researchers exploring the intersection of generative AI, mathematics, and cognitive science. We analyze how architectural design impacts learning, memory, and reasoning in foundation models.
Join us to dissect training dynamics and explore how cognitive principles can inform the next generation of architectures. Our next session will be hosted via Google Meet on: Monday, August 10 · 12:00 PM AEST (Melbourne) which is Monday, August 10 · 10:00 AM CST (Beijing) which is Sunday, August 9 · 7:00 PM PDT (San Francisco)
Apply to join HERE: i14.ai/journal-club/













