Founder @ Rafter | How do minds and machines solve intelligence?
Berkeley, CA
Joined November 2017
- Tweets137
- Following51
- Followers94
- Likes43
Pinned Tweet
I started @RafterSecurity as the security layer for AI agents.
But it's also a free, open source toolkit that instantly improves the security of everything they do and create.
With one prompt.
@openclaw @ship_builders @composio @nebiusai @tavilyai
So...apparently there's no such thing as meter in French?
It just doesn't have lexical stress. I pity parisian poets.
I scored three ways: CLIP , DINOv2, DreamSim.
*CLIP and DINO agree at ~150 words (the 155 in the figure is a binning artifact, not a coincidence). DreamSim—which measures perceptual alignment rather than a learned embedding distance—sits nearer 350.
Truncation (dropping the end of long descriptions) doesn't flatten the curve. It inverts it.
Naively pruning description lengths isn't reliable. The opening tokens of a long description are a worse prompt than 55 words written to be 55.
Long descriptions open with scene-setting and spend detail later—truncation keeps the throat-clearing and discards the substance.
Training caption length distributions are hard to pin down—most labs don't publish them—but these plateaus sit at reasonably lengths, so it's probably not an artifact of training.
Code: github.com/Rome-1/words-per-…
the age old question: is a picture worth 1,000 words?
The idea is to use text-image models to go image -> text -> image and then use a loss metric between the start and end image (like CLIP cosine). Then compare across different description lengths.
Turns out it's
about 150 to 350.*
Reading tertiary structure (residue contacts) out of a protein language model usually means the Categorical Jacobian (Zhang et al. 2024): ~20L forward passes.
You don't need them — the contacts are already in the attention, and one forward pass recovers them.
It also scales past the context window: tile + stitch overlapping windows (near-lossless), or use a longer-context backbone (ProtT5, ESM-C). Two real L>1024 targets read in ~1s of forwards, where a tiled Categorical Jacobian is prohibitive.
Paper: arxiv.org/abs/2606.21876
Code, figures, and result tables: github.com/Rome-1/plm-contac…
Taught Qwen2.5-3B to speak Yoruba: 59.6% drop in perplexity on held-out Yoruba (18.8 → 7.6).
Unsurprisingly:
- Localizing English instructions into culturally-grounded Yoruba beat literal translation as a data recipe
- Belebele MCQ accuracy showed ~0 gain; at this scale it's near-chance and blind to the improvement
Built for @adaption_ai's AutoScientist challenge with their Adaptive Data API.
Open-source — adapted dataset + trained weights on HF & Kaggle:
Model: huggingface.co/Rome-1/qwen2.…
Dataset: kaggle.com/datasets/romethor…
Do you trust agents more for precision or recall? I’m falling into a weird recall trap.
Let me explain.
I’ve been editing a lot of AI-assisted papers recently. And I’ve noticed that, by the time I get to the papers, I assume everything worth fixing has already been caught—that the only things left to scrutinize are the changes themselves.
So I’m implicitly assuming the agents have over-flagged, or mistakenly flagged, things. I’m checking the edits carefully, but I’m not checking the rest of the draft nearly as much.
It’s the same thing when reviewing a PR. There, usually it’s safe to trust the rest of the code and focus on the diff.
But there’s this funny middle ground in AI-led content when you’ve had a huge and careful role in editing the initial draft. Then you make a few rounds of sweeping changes. Then you start looking very carefully at each new change. Presumably, you and your co-authors are also checking everything carefully—but it becomes very easy to just accept the diffs in the middle.
So that leaves us with a strange asymmetry: I don’t fully trust that the agents’ flags are correct, but I do trust that they haven’t missed anything.
In other words, I’m skeptical of their precision while quietly assuming near-perfect recall.
That feels like a dangerous way to review anything.