making coffee and other things. @BrewTimerApp
boston
Joined May 2013
- Tweets905
- Following25
- Followers545
- Likes1.4K
I’ve seen both traditional and AI-powered accounting firsthand at Digits, and it’s obvious to me where things are headed. Excited for this partnership with CLA! They are the real deal.
Today, @CLAconnect CliftonLarsonAllen — one of the largest accounting firms in the country — announced that they've selected @digits as their AI-native accounting platform and will deploy our ledger to many thousands of their clients over the next 3 years.
Excellent rundown of the current state of LLM quantization.
We are doing really cool hard tech at @trymirai, but until recently our social media feeds were full of linkedinish cringe. We decided to fix it and share more technial content
I am currently working on our quantization pipeline, so here is a thread about LLM quantization
This ButterflyQuant paper looks neat, but also a little sus:
- no code
- no comparison against its closest relative (SpinQuant)
A good test project for coding agents?
Incoming new coremltools looks like it has some nice bits:
- 8 bit input/output tensors (previously all 8bit compute was kept internal)
- >1 input can be enumerated shapes (👀ANE)
Turns out you don’t need R₅⁻¹ at all. 🫠 Fusing into Q and K is enough!
Cool paper from Qualcomm explains this and a few similar transforms.
No code in the paper, so gist proof👇
Liking the line of research where you multiply LLM weights by rotation matrices and the model still works.
Most do it in between layers, but you can also sneak one between Q/K and RoPE.
Extra parameters? None.
Useful? …Maybe.
Cool? I think so.
(See R₅ below.)
ALT Attention portion of the SpinQuant diagram, with R₅ added.
🐙: gist.github.com/smpanaro/5d3…
📄: arxiv.org/pdf/2506.04985
(R₅ is a rotation matrix, so its transpose is its inverse and it naturally cancels out in Q@K.T)
Curious about the Apple Foundation Model architecture? I updated my netron fork to visualize the draft model*.
*they say it might differ from the real model but looks convincing to me
See for yourself:
1. Get the adapter training toolkit: developer.apple.com/apple-in…
2. Clone: github.com/smpanaro/netron/t…
3. Edit draft.mil:
- delete all functions except the first
- rename it to: func main<ios18>(
4. Follow readme to start netron, and open the .mil
The python library is interesting too. “Download files”:
pypi.org/project/tamm/0.1.0/
Cool to see folks measuring KL too.
arxiv.org/abs/2505.22988
Looks like a lot. The weights are there (in 32bit) and there’s a python package to load them.
Either way, wonder how much we can learn about the model from this.
developer.apple.com/apple-in…
