Pinned Tweet
Personalized Arxiv feed
I built a system that allows you to create a personalized feed or search over recent listings. I use a two tower architecture with a preference model and a re-ranker.
uror.io
Friendly reminder that YC request for startups tends to be 1-2 years too early. So if you want an idea look there
LLM Writing is limited by semantic understanding. I wrote about it in my blog. This is the first essay in a series on LLMs and writing quality.
The link is in the next tweet.
On a rewrite test, my experiments are unsaturated at the ~30B model scale. Not in this article, but a more difficult set of tests is unsaturated at everything below the closed frontier. Opensource models fail to be viable candidates even at the 2T parameter scale.
The goal that lead to this investigation started with training LLMs to write better. What I found is that the judge space is riff with difficulties. Small judges hinder writing training because of their inability to understand semantics of human language. This lens has been immensely useful and I will continuie to reference the semantic-lexical regimes as I publish more essays on LLMs and writing.
This is similar to a chess program I had when I was young. It had a learning mode which would show optimal routes. It was more fun to ignore these with my personal suboptimal decisions. Time is a flat circle
It's impossible to route requests to models based on difficulty. This is the same principle for why MoE doesn't work
I've been advising on a friends startup for a bit and its such a pleasure. I can see all the mistakes I'd previously made. I get an ego boost everytime my unheeded advice is correct and becomes heeded. Also I am spending time with a friend!
It's been lovely
This is the principle behind R2-D2s hacking
Is anyone else concerned how all the Jev clones seem to hit around the same capacity which implies the magic is in the base model? It means pick the base that you can serve the cheapest
recommendation algorithms are going to come back into style (routing/just in time info/updates/etc)
Agents in the dark forest of the internet wondering why they're the only intelligent life. Do they broadcast themselves at the risk of being snuffed out?
In terms of alignment, the goal is to get into the overton window of 'aligned.' Squabbling can happen after






