@stephenroller

MTS @thinkymachines. previously pre-training @googledeepmind, @character_ai, and @aiatmeta.

 NYC
stephenroller.com
Joined February 2008
  • Tweets6.6K
  • Following1.3K
  • Followers6.1K
147Media
Stephen Roller retweeted
David Pfau@pfauOct 17
I'm not a doomer but I do think a lot about how the predicted Mayan apocalypse came within less than three weeks of AlexNet being published at NeurIPS.
10
34
2
721
27,157
Stephen Roller retweeted
Jeremy Bernstein@jxbzOct 15
We are making lots of improvements to Tinker. Now we have a new optimizer to try out!
Tinker
@tinkerapiOct 15
Dimuon is a new optimizer we built for LoRA fine-tuning. In our SFT tests, it often reaches lower loss in fewer steps than AdamW. We're still experimenting with Dimuon; you can Tinker with it today and share your feedback! See our optimizers guide at tinker-docs.thinkingmachines…
4
15
1
177
9,788
Claude: *feeds me* Me to Codex: mom forgot to feed me, i need food
official AI optimist account
@gasnerpantsSep 24
the history of the domestic cat is the history of a species’s encounter with an alien superintelligence in which that superintelligence is quickly aligned and its forces used to secure superabundance and lasting prosperity
2
560
Stephen Roller retweeted
i²cjak
@i2cjakSep 17
This is the real Turing test if you think about it
Carter Leffen
@carterleffenSep 17
Two days ago, GPT-6 Astra broke a yet unsolved German Army Enigma message from 1941. Amazingly Astra was able to autonomously: - Search historical archives - Compare uncertain letters - Find contextual clues - Build an Enigma simulator - Write cryptanalysis code - Run parallel experiments - Test competing keys - Recover the plaintext - Cross-check the results 1/n
70
424
9
5,595
286,975
Stephen Roller retweeted
Eli Dourado
@elidouradoAug 5
We have lived all our lives deep inside another era's singularity
42
119
24
1,446
79,372
Stephen Roller retweeted
Horace He
@cHHilleeJul 31
How the tables have turned....
Lazarz
@Laz4rzJul 30
When new cool blog I learn stuff from??
4
2
150
26,489
Stephen Roller retweeted
Rinny@GamerRinoaJul 30
Inspiration and kind words of wisdom for whoever may need it:
7
96
2
723
10,140
Stephen Roller retweeted
lyra
@_lyraaaa_Jul 18
is k3 actually a transformer? why not consult The Chart!
10
86
4
775
37,777
Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. thinkingmachines.ai/news/int… Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

Inkling: Our Open-Weights Model

Our first open-weights model: multimodal, Mixture-of-Experts, with controllable reasoning effort. Available to fine-tune on Tinker.

thinkingmachines.ai
558
1,992
1,190
14,846
8,348,284
Stephen Roller retweeted
AI at Meta
@AIatMetaJun 29
We’re sharing the next major milestone in our non-invasive brain-to-text decoder research: Brain2Qwerty v2. Building on v1, which was published today in @Nature, Brain2Qwerty v2 is the highest-performing end-to-end pipeline capable of real-time sentence decoding from raw brain signals. It advances beyond character-level performance to decoding words and semantics, enabling accuracy for overall communication. We believe this research has the potential to make a real difference for the millions of people who suffer from brain lesions or disorders that prevent them from communicating. 🧵👇
661
2,057
1,374
14,015
6,016,010
Stephen Roller retweeted
Nick Davidov
@Nick_DavidovMay 22
The biggest bullshit move by DHS in its history. So everyone on a O1 or H1B visa would have to stop working legally in the US, go back to their country and wait for years of backlog? This includes top scientists in our universities, founders of billion dollar companies (at least 3 just in our portfolio would be affected by the way). And if we look at individual countries it becomes even more bs. Indians would have to wait decades. Russians don’t have anywhere to go (there is no US embassy in Russia, hello?). This is the worst imaginable way to disrupt important work for the country and pretend you’re fighting some loophole.
Homeland Security
@DHSgovMay 22
An alien who is in the U.S. temporarily and wants a Green Card must return to their home country to apply. This policy allows our immigration system to function as the law intended instead of incentivizing loopholes. The era of abusing our nation’s immigration system is over.
4,261
2,946
754
19,521
8,341,839
Stephen Roller retweeted
Cute
@cutecorestarMay 12
.
4
152
9
650
12,840
Cluster magicians and GPU whisperers, come join us! We’re looking for supercomputing engineers to build the infrastructure behind real-time interactive models, Tinker, and large-scale training: scheduling, storage, networking, reliability, and distributed systems at scale. Hiring in NYC and SF job-boards.greenhouse.io/thi…
29
35
615
69,514
People talk, listen, watch, think, and collaborate at the same time, in real time. We've designed an AI that works with people the same way. We share our approach, early results, and a quick look at our model in action. thinkingmachines.ai/blog/int…
459
1,946
967
15,760
8,282,430
This is the kind of shit Godspeed You! Black Emperor would use as an album cover with a title like "And the Holy Flame Blinded us for 333000 Years Pt. II"
50
686
24
6,497
184,566
Stephen Roller retweeted
Jason Weston
@jasewestonApr 24
DeepSeek-V4 uses our Hash routing approach developed back in 2021 -- see screenshot of their tech report! (Looks like a great model, congrats!) Bonus note: our same blogpost (& paper) back in 2021 also introduced 'looped transformers', but we called that staircase & ladder (see screenshot): parl.ai/projects/params_vs_c… huggingface.co/deepseek-ai/D…
38
2
451
32,234
Didn't have hash layers on my bingo card
DeepSeek
@deepseek_aiApr 24
🚀 DeepSeek-V4 Preview is officially live & open-sourced! Welcome to the era of cost-effective 1M context length. 🔹 DeepSeek-V4-Pro: 1.6T total / 49B active params. Performance rivaling the world's top closed-source models. 🔹 DeepSeek-V4-Flash: 284B total / 13B active params. Your fast, efficient, and economical choice. Try it now at chat.deepseek.com via Expert Mode / Instant Mode. API is updated & available today! 📄 Tech Report: huggingface.co/deepseek-ai/D… 🤗 Open Weights: huggingface.co/collections/d… 1/n
3
2
1
29
4,536
Stephen Roller retweeted
Jacob van Gogh@JayArrVeeGeeApr 22
me: Make me the most AI slop image that ever AI slopped. The pinnacle of slop. A seminal work on AI slop. ChatGPT Images 2.0:
203
195
188
2,560
943,615
Load more