Pinned Tweet
🚀 Abacus Desktop 0.1.0 is out.
The abacus coding agent, now in a window: your projects and sessions, the agent's work as it streams, the diff, a terminal, and a browser it can use.
Open source. macOS, Windows, Linux. Any model.
abacus.empero.org
One command installs it. The installer asks whether you want the app too:
macOS, Linux:
curl -fsSL abacus.empero.org/install.sh | sh
Windows:
irm abacus.empero.org/install.ps… | iex
abacus.empero.org shows the right one for your system.
Abacus 0.7.0 ships with it: abacus update keeps the CLI and the app current, and the app server gained the APIs behind background tasks and browser use.
Both are open source:
github.com/empero-org/abacus…
github.com/empero-org/abacus
We are still actively looking for people to help us build!
Many of you have reached out wanting to contribute to Empero. Until now we had no good way to say yes.
So we built one: the Empero contributor program.
Research, data & evals, systems, community and partnerships.
empero.org/hiring
Setup: small decoder, equal-size tables (200M params), 5 seeds sharing init and data order.
After 60M tokens PageGram beats a flat table by 0.067 nats and Engram-style by 0.142, on every seed.
With 2x the tokens it is level with Engram-style and ahead of flat on average.
Stored at 4 bits (NF4, costs 0.0004 nats), a 20B-parameter table with a 1 GiB RAM page cache needs 0.99 flash reads per token, against 9.18 for Engram-style addressing. A cached PageGram page serves every head of every n-gram on it.
From NVMe: 5–8x as many lookups per second.
PageGram is the n-gram memory of our current models.
Paper, harness, C flash simulator and the results of every run (Apache-2.0):
github.com/empero-org/pagegr…
PageGram keeps the heads but puts them in one page. The n-gram picks a page, and each head picks one of S slots inside it.
A token reads 2 pages, as with a flat table. Per head, rows are exactly as crowded as with Engram-style addressing.
🧵 3/7
New from Empero: PageGram: Multi-Head N-gram Memory at the Read Cost of a Flat Table.
Engram-style n-gram tables cost 16 random flash reads per token. PageGram reads 2, and matches or beats both flat and Engram-style tables in loss.
📄 alphaxiv.org/pdf/2610.pagegr…
🧵 1/7
New from Empero: Identity on Demand: Multi-head Gated Value Residuals (MGVR).
A drop-in change to attention that reaches the same loss in 16–22% fewer steps: 1.15–1.23× effective compute for +0.2% parameters.
📄 alphaxiv.org/abs/2610.multi-…
🧵
Why it works, on synthetic tasks (6 variants, 3 seeds each):
Induction heads: 40% less time from attention pattern to accurate copying.
Two-hop recall: learnt 25–49% faster than a Transformer and 7–18% faster than ResFormer.
The token-dependent head mixing does the work.
Drop-in for any MHA or GQA layer. It starts exactly as ResFormer, leaves the attention kernel and KV-cache size unchanged, and takes ~20 lines of PyTorch.
Code, results and paper source (Apache-2.0):
github.com/empero-org/mgvr
