Huh: this is a 48B model that itself won't do much, but efficient attention that works well'd be pretty nice for "let's have a chat re: these 10k lines of code" and such
Context Arena Update: Added kimi-linear-48b-a3b-instruct [11-08] and kimi-k2 (Thinking) [11-06] to the MRCR leaderboards.
The Linear 48b results are fascinating! It actually outperforms the new Gemini 3.0 Pro Thinking on 4-needle and 8-needle tasks at higher context lengths (512k+). I've added it to 2needle, 4needle, and 8needle.
kimi-k2 (Thinking) lands lower on the leaderboards (Rank #22 for 2-needle AUC @ 128k), with a hard context ceiling around 262k. I did not run it for 2needle and 4needle.
All results at: contextarena.ai
The performance curve for the Linear model is distinct: while it underperforms Gemini 3 significantly at shorter contexts (<=256k) on the difficult 8-needle test, its degradation slope is much flatter. Gemini starts higher and drops fast; Kimi starts lower but holds steady, overtaking Gemini at the higher end.
However, note that kimi-linear-48b has noticeable performance drops past 128k on the easier 2 & 4 needle tests. Additionally, due to lower token efficiency compared to Gemini/GPT, only ~60% of the 1M token tests successfully ran (hitting limits/OOM). So some caution with the results at the 1M level.
kimi-linear-48b results:
2-Needle Performance (@ 128k / @ 1M):
- AUC: 96.5% (vs Gem 3: 99.5%) / 81.7% (vs Gem 3: 85.5%)
- Pointwise: 96.0% (vs Gem 3: 99.0%) / 77.0% (vs Gem 3: 72.2%)
4-Needle Performance (@ 128k / @ 1M):
- AUC: 85.5% (vs 85.8%) / 62.7% (#1, beating Gem 3: 57.3%)
- Pointwise: 83.7% (vs 80.8%) / 51.5% (#1, beating Gem 3: 34.3%)
8-Needle Performance (@ 128k / @ 1M):
- AUC: 54.9% (vs 73.0%) / 43.8% (#1, beating Gem 3: 39.0%)
- Pointwise: 49.0% (vs 54.2%) / 35.3% (#1, beating Gem 3: 24.5%)
A very different architectural approach yielding impressive stability at scale. Because of its current price point, it is very competitive for long context (MRCR).
Enjoy.
@Kimi_Moonshot
@GoogleDeepMind @googleaidevs
@OpenAI @OpenAIDevs
rf retweeted
A mysterious expert in submersibles was interviewed by the Coast Guard during the Titan investigation. His name is redacted, but we barely get into the interview before it becomes obvious who it is.
rf retweeted
Got sent this clip from the Iranian comedy show "Bachelors". Subtitled it because I thought it was hilarious
rf retweeted
they should let you set your phone screen to grayscale except for one person you really like, whose posts display in color
rf retweeted
Some Well-Meaning Knaves Have Asked Innocently Enough (And So I Generously Ignore Their Knavery) What Level Of Detail I Hope To Achieve With My Scans. Here I Present A 15,000 Gaussian Per Square Meter Scan Which Is My Ambition To Complete For The Entire City God Above Willing
I mean, the *dream* is for SCOTUS to rule...7-6 in favor of the next try at the program, instead of 6-3 against
rf retweeted
German discount grocer Lidl built their own cloud infrastructure because of German data laws, and now has a ~$2B revenue business selling it to others (a la AWS)
rf retweeted
Already lots of qrts about how the ancients had spreadsheets too but it really needs to be emphasized that their sheets look *exactly* like ours
rf retweeted
so i'm at the central state museum of Kazakhstan, and they have this dombra exhibit. i notice that one of them is modern-looking and out of a case, so i ask one of the docents if i could try playing it. i sit down & start strumming away, basically playing it like a balalaika
one of the other guests, a middle-aged Kazakh guy there with his family, is watching me, and i can recognize the antsy look of someone watching someone else butchering their instrument, so i hand it to him
he pops off his shoes, sits down, and starts *wailing* on it
after two songs, the docent is like "wait, sit here" and slides this table over to him, which has this little string attached. he slips it around his pinky, and…
🎶🐐🎶
