Building a universal predictor for applied ML, data science, and annotation.
aquinlabs.com
Joined July 2023
- Tweets146
- Following6
- Followers172
- Likes666
We fine-tuned a talking-avatar LoRA on MiniMax-H3 Ref2VA: one character image plus dialogue becomes a clip with model-generated speech, lip sync, and light motion.
On ~385 clips with just 10.6M trainable parameters using aq. It handles portraits, paintings, and anime characters!
Model: huggingface.co/Aquinlabs/h3-…
Code: github.com/Aquinf03/h3-chara…
Launching MoM: Mixture of Models.
Large multimodal models are powerful but expensive to run. MoM is a cost-efficient alternative: compose smaller, specialized models and tools into one pipeline, and call the whole thing like a single model.
Build AI layers by combining different edge models (vision, speech, SLMs, decision, classifiers etc.) with Shared state, speculative routing and a ready-made API come built in, and the latency-critical path runs on Rust!
Code: github.com/Aquinf03/MoM
A single image becomes a talking avatar. @MaxMill06 trained H3 Avatar on ~385 clips with just 10.6M trainable parameters using aq.
It handles portraits, paintings, and anime characters with generated speech and lip sync
I trained H3 minimax for talking Avatars using @AquinF03
Turn a single image into a talking avatars
Aquin Labs retweeted
ash is 17 and has already started over five times.
for him, the biggest support is his mom who has said yes more times than we can think of.
now he's doing interpretability research and running @AquinF03 labs
he was the youngest in our last cohort and has been hitting miles. so proud of this one!
New paper: Attention as a Routing Graph: Live Circuit Extraction from a Single Forward Pass.
Finding circuits in LLMs usually means running tons of careful interventions, expensive, slow, one hypothesis at a time. We tried something simpler: treat attention as a routing map from a single forward pass, keep the routes pointing toward the answer, and test whether they actually matter.
Cost: one forward pass. A full patch sweep costs roughly 100x more.
arxiv.org/abs/2609.25285
Aquin Labs retweeted
Published a new paper: Attention as a Routing Graph: Live Circuit Extraction from a Single Forward Pass.
Finding circuits in LLMs means running tons of careful interventions. We tried something simpler, treat attention as routing map from just one forward pass, keep routes that point towards the answer, and check if they matter. On many tasks ablating our extracted routes hurts the model way more than ablating random set of same size.
Costs one forward pass, versus roughly 100x more for a full patch sweep.
arxiv.org/abs/2609.25285
21M-param world model for a pixel-art fox game.
@MaxMill06 built a custom tokenizer and planner from scratch, trained on 2M+ pixel-art gameplay frames, on a model that takes player actions and predicts the next game frame.
Repo: github.com/Aquinf03/Pixel-Ar…
Model: huggingface.co/Aquinlabs/pix…
Dataset: huggingface.co/datasets/Aqui…
We love seeing what our team builds and experiments with our tooling
a full world model with its own tokenizer and plannertrained on 2M+ pixel-art frames, into a playable game!
I trained a 21M-parameter world model for a playable pixel-art fox game using @AquinF03
Built from scratch with:
- A custom tokenizer and planner
- 2M+ pixel-art gameplay frames
- A model that turns player actions into future game frames
New research: Stereotype Feature Steering
Can stereotypical behaviour be traced to individual features inside a language model?
In this work, @sheeeevam investigated whether occupation-to-gender stereotypes could be localized to specific SAE features in Llama-3.2-1B-Instruct.
We identified a single feature strongly associated with the model’s tendency to complete “The mechanic said that…” with “he” rather than “she.”
Steering this feature moved the model’s behaviour toward a balanced 1:1 distribution, and could even override an explicit gender cue in the prompt.
We also found that the same stereotype was encoded differently in LFM2.5-230M, suggesting that similar behaviours did not necessarily correspond to the same internal representation across models.
Read the full research: aquin.app/research/simulatin…
we’ve published our first paper: Finding Usable Weight Mechanisms with Tiled SVD.
most interpretability methods learn a separate dictionary(like SAEs) to identify features in a model.
we instead extract mechanisms directly from the weights using tiled SVD, giving each mechanism a clean trigger, write direction, and strength.
evaluated on Gemma-2-2B: 182/182 evaluations passed.
paper arxiv.org/abs/2608.06969
Aquin Labs retweeted
Published my first paper: Finding Usable Weight Mechanisms with Tiled SVD.
Mostly interpretability work trains separate dictionary (like sparse autoencoders) and finds features by what text lights them up, but that "concept" lives in the dictionary, not the model weights.
We skip that and pull mechanisms straight out of weights using tiled SVD, each one is a clean trigger, write, and strength without dictionary.
Tested on Gemma-2-2B with full eval suite pass 182/182.
arxiv.org/abs/2608.06969
we simulated LLMs in chess, gossip and Among Us sandboxes.
and used aq to inspect, find behavioural features with SAEs, steer them and we could watch the behaviour change.
also benchmarked their reasoning in real-time
Literature: aquin.app/research/simulatin…
In chess, we looked at the model’s activations alongside the game state.
the model had to make a move and explain it, while aquin let us poke around the activations behind both.
so we could look at:
what the model represents → what it says → what it does
and see where those three line up.
then we took the same setup into echoes and among us.
we instrumented the models across entire games and watched their internal states change as they talked, lied, shared information and tried to figure each other out.
basically, we wanted to see what’s going on inside while the models are actually doing stuff.
We built mechanistic interpretability tooling that runs on a local GPU. Load an LLM or embedding model, then inspect, simulate a train, diff checkpoints, steer, watch a live run, and eval. SAE capture, train, and align; weight and residue diffs; red-team and custom evals. The sections below are the original applied-research writeups, kept in full.
Repo: github.com/Aquinf03/AQIT
Article: aquin.app/research/interpret…
