Financial economist. Markets, AI, asset pricing. Mostly harmless.
Joined October 2019
- Tweets284
- Following771
- Followers252
- Likes2.3K
Pinned Tweet
🚀 Introducing ChronoBERT: A Chronologically Consistent Language Model ⏳📜
Excited to share ChronoBERT, a joint work with @LinyingLyu , @AsafManela , and @jimmywucm ! Our new pre-trained language model ensures no lookahead bias and strong language understanding! 🧠✨
Songrun He retweeted
what objective is reasoning solving?
i've been trying to better understand some weird behaviors in RL/reasoning (especially latent reasoning), so i went back to basics to correct some misunderstandings i had
new post on reasoning as pathfinding in token space:
Songrun He retweeted
Anthropic 路演视频太炸裂了吧！谁说 AI 有泡沫的？
开场，Dario Amodei 预言：癌症、阿尔茨海默病、心脏病、精神疾病，"不到十年"会成为过去。
路演里 CFO 给出的数字：
· Q3 营收近 170 亿美元，同比 12 倍
· 今年前三季度毛利率 85%，和美光最新一季的 86.8% 基本持平，而美光是靠这轮存储涨价才冲到这个水平的
· 工程师人均代码量是 2024 年的 45 倍
个人看法：码农真的是夕阳产业，大科技公司还有很大的裁员空间
· 人均营收从 140 万美元升到 660 万美元以上
· 超过一半的财富 500 强年投入超过 10 万美元，但平均只占它们总支出的 0.1% 不到
按路演的说法，现在赚的钱只是冰山一角
· 年投入超过 10 万美元的客户 11,900 家，比年初增长近 8 倍
客户：BMS 预计临床试验时间缩短 20–30%；埃森哲的安全扫描从几天缩短到不到一小时。
结尾，Dario 说"如果我们做对了"，就能开创一个"奇迹时代"。
不构成投资建议。
$ANT
Songrun He retweeted
机器人世界模型赛道完蛋了，Fable 5.2 已经用基模给搞定了
The bitter lesson 又来一遍
Songrun He retweeted
i repeat, specialist models are a dead end
Songrun He retweeted
In one of my favorite experiments, I told Fable 5.2 to build an endless, hands-off journey through 3D worlds linked by portals.
25,000 agents created over 450 worlds inspired by books, art, places, all in code.
Play: portals-endless-worlds.netli…
Github: github.com/emollick/ai-porta…
Songrun He retweeted
18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over.
And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams.
So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit:
- how many experts, how big, shared or not?
- why is half my step spent in all-to-all, and how do I hide it?
- why is my run 25x slower with no error in the logs?
- softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32?
- aux loss or DeepSeek's loss-free bias?
- upcycle a dense checkpoint or train from scratch?
- can I serve with fewer experts per token than I trained with?
⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs.
Read it here: huggingface.co/spaces/huggin…
Songrun He retweeted
📖 The Ultimate MoE Training Handbook 📖
We spent 1/2M GPUh to optimize MoE training and wrote the most comprehensive blog about it. With 30k words and 50+ interactive figures it covers everything from profiling to large cluster training.
huggingface.co/spaces/huggin…
Songrun He retweeted
Don't get too confident about prediction markets! MIT finance department PhD job-market candidate working on prediction market shows miscalibration. The long-shot bias exists and exploiting has a historical Sharpe ratio in the neighborhood of 2!
jiagengliu.com/
Songrun He retweeted
maybe a hot take but i now consider adversarial reviews mostly a waste of tokens. far better to use those tokens on empirical combined with formal verification. i have updated pstack to use the interrogate skill less, and eventually i think i may remove it altogether
i have found very little value with code review in the abstract, agents tend to just get lost in the nitpicks and yagnis
Songrun He retweeted
This is so insanely misleading.
No, it did not edit the raw weights. It used existing tooling to identify a problem area for the model, then it found the direction associated with it, and ablated that from the model.
The oversell here is downright intellectually dishonest.
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Songrun He retweeted
(wild) predictions after today's MIT Weight-Space Programming work:
1. Model weights become a programming language that AI can understand, but humans can't.
2. Training and inference will merge. Like human brain, AI could continuously rewrite its own weights while thinking.
3. A new path to RSI: AIs will directly edit each other's weights, evolving together without backprop.
Songrun He retweeted
Weekend hack: HackerTube. I was annoyed with YouTube's recommendations so I had Claude filter down 43M comments over 20 years of HN to 6,454 videos that keep being recommended. *Significantly* better viewing experience, ~2 hours with Opus 5.5:
hackertube.org/
- Excellent recs without any logins (state is kept in local browser session)
- Thumbnail is a representative frame from middle of the video instead of the super duper A/B test hyper-optimized user uploads
- Underlying data from the comment analysis as one zip file, point your Claude at it and ask for your own version
- HackerTube Longs ✊
A bit more on the process:
I use Claude Code (desktop app for Mac), Opus 5.5 on Medium effort as my default. I only talk to Claude and have it do everything, the only "manual work" I helped with in tandem was in the browser (reminiscent of MenuGen): 1) I had to create some kind of API key from Google Cloud UI to query YouTube for video metadata, 2) I booted up a VPS box for serving and 3) I bought the domain name. Once the box was up I gave Claude the IP and told it update, secure, harden, deploy and optimize the code, and it did the whole thing.
Many awesome projects that would have normally taken weeks/months even can now be few-shotted for breakfast. What else are we not thinking of.
Songrun He retweeted
🚨 The 2026 Open Source Asset Pricing data release is live! We migrated to CRSP v2, moved the data to Hugging Face, and updated the Python and R download packages. As in every release, the updates show our commitment to having the highest quality asset pricing dataset out there 🧵
Why fix token boundaries upfront when the model can learn them end-to-end? Exciting direction!
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
1
66
Building AI agents is getting easier. Evaluating whether they actually work is still hard.
Stanford’s CS329Z looks particularly useful for its emphasis on evaluation, reliability, and agent design.
强烈推荐斯坦福的这套关于 AI Agent 的课程，对于有志成为 agent engineer 的同学可以看看：cs329z.stanford.edu/
2
3
433
