Ramakanth Kavuluru retweeted
If you've noticed a distinct dialect emerging in frontier AI models and struggled to understand what they're talking about, you're not crazy!
We turned to psycholinguistics to investigate what makes the writing style so weird, and why this might be happening
🚨New preprint! 🧵
Ramakanth Kavuluru retweeted
Introducing Carbon-X: The best open DNA model.
We midtrained a 14B model on DNA and text, and it beats every open language model and matches GPT-6.1 Sol at 1,000x lower cost.
Ask it about a sequence, what a mutation does, or to generate new DNA.
huggingface.co/HuggingFaceBi…
Ramakanth Kavuluru retweeted
Today we're releasing Carbon-X, a 14B model that brings DNA and natural language together for the first time 🧬
1. Beats every open language model we tested across DNA benchmarks. Within a point of GPT-6.1 Sol at >1,000× lower inference cost.
2. Mid-trained on 300B tokens of interleaved DNA and text, followed by SFT (without distillation!) and RL.
3. A new hybrid tokenizer encodes DNA using ~3× fewer tokens than BPE.
4. Gives other models a DNA specialist subagent: Kimi K3 gains 11.1 points when it consults Carbon-X.
5. Runs locally on a laptop with llama.cpp. Apache 2.0. Your DNA stays on your hardware.
Model: huggingface.co/HuggingFaceBi…
Blog: huggingface.co/spaces/Huggin…
Introducing Carbon-X: The best open DNA model.
We midtrained a 14B model on DNA and text, and it beats every open language model and matches GPT-6.1 Sol at 1,000x lower cost.
Ask it about a sequence, what a mutation does, or to generate new DNA.
huggingface.co/HuggingFaceBi…
Ramakanth Kavuluru retweeted
💡 Training on auxiliary views of Document X > training on Document X itself, even for factual questions drawn directly from X.
Presenting our work on LLM knowledge acquisition at #EMNLP2026 this weekend!
(1) Under a fixed token budget, conceptual reformulations (textbook explanations, blogs, and Q&A) can teach a document’s facts more effectively than repeating the document, even paraphrased.
We call these auxiliary views: beyond mere paraphrasing, they discuss the knowledge in a completely different context and form e.g. a paper blog!
The surprising part is that auxiliary views rarely mention the facts directly, but our evaluation uses the original document's phrasing near-verbatim.
(2) The benefit emerges and grows with model scale.
(3) stronger generator ≠ better learning.
We synthetically generate auxiliary views for our experiments, but across GPT-5, GPT-OSS, Gemma4, GLM-5 and more, the learner model's knowledge acquisition shows no correlation to generator model size or its own accuracy on the evaluation.
(4) After training on auxiliary views, the model's parameters actually move less but concentrate its changes at the middle and final layers.
The takeaway for synthetic data augmentation: build diversity around the knowledge you want to teach, instead of treating diversity as some board corpus-level heuristic.
Also, a possible implication for LLM self-improvement. Models can learn by just generating and training on their own auxiliary views, without a stronger teacher. This is particularly relevant for deeply specialized, long-tailed knowledge.
📄 Paper: arxiv.org/pdf/2609.04180
Ramakanth Kavuluru retweeted
🚨 New Paper! 🚨
Multiple-choice benchmarks use accuracy (boring), but education has way cooler scoring schemes! 🤩
👎 Lose points if wrong
🔁 Try until you're right
🃏 Wager points
At #EMNLP2026, we show these schemes shift LLM ranks, align with LLM Arena, + unmask LLM quirks
Ramakanth Kavuluru retweeted
Super excited to share SnapJudge -- a free, open detector trained for scientific and math-heavy writing, and can handle multiple modalities, including screenshots of paper text, LaTex, and text extracted from PDFs... 🧵
Ramakanth Kavuluru retweeted
Can research agents get better at science by learning from the research they do?
Introducing DiscoveryRSI 🧬: a framework that turns hypothesis-driven research into a recursive self-improvement loop.
📄 Paper: arxiv.org/abs/2610.23839
💻 Code: github.com/Gen-Verse/Discove…
🔴 Live Arena: foundry-arena.rsi-arena.work…
🔹 What is a valuable hypothesis?
We define it as a joint object H = (claim, investigation plan, decision criteria, alternative explanations). A hypothesis is valuable if resolving it would change a research decision. It is testable if the plan can produce evidence that supports, weakens, or bounds the claim.
🔹 How it works
- Lead, Generation, Evolution, Critique and Progress agents grow hypothesis trees through evolve / repair / crossover / restart
- Every version stays linked to its evidence, data, assessments and contribution records
- Selected plans compile into executable Harbor tasks
🔹 Three self-improvement mechanisms
① Hypothesis credit optimization: blinded TrueSkill assessments assign bounded credit to the agent steps that actually contributed, and DiscoveryRSI-27B is trained only on those steps (Qwen3.8-27B init, 256K context; 2,397 of 15,792 trajectories kept; 2.12% of tokens supervised)
② Harness replay: diagnose recurring failures such as repeated exploration, failed tool use, lost artifacts and premature closure, then refine the workflow
③ Rediscovery: run the accepted model and harness updates on fresh questions
🔹 Results (DFM-Bench, 4 life-science domains)
- Hypothesis quality 4.86 → 5.18 → 5.39 over two rounds
- Falsifiability 4.00 → 6.20, alternative explanations 4.44 → 5.98, data resolvability 3.73 → 4.99
- Evolve/repair share 77.5% → 91.7%, restarts 21.7% → 5.0%: the agent learns to keep developing an idea instead of starting over
- Caveat: importance and novelty dip slightly. The model learns to design better tests, not yet to choose bolder questions.
🔹 Expert review
27 physicians and bioscience researchers scored 292 anonymized proposals:
- 41% of DiscoveryRSI-27B revisions improve quality vs 31% (GPT-5.6 Sol) and 25% (GPT-6 Luna)
- Only 12% make things worse vs 32% and 31%
🔹 Does it anticipate real science?
Against 16,440 publications filtered by each model's cutoff, 6.3% of DiscoveryRSI-27B's hypotheses connect to later published research (16.9% in therapeutic targets). Comparison: DeepSeek 5.4%, GLM 2.8%, GPT 2.5%, Claude 2.3%.
Watch six models do research in real time in DiscoveryRSI Arena, and follow every proposal, critique and compile step 👇
foundry-arena.rsi-arena.work…
Co-led by Jianyuan Zhong, Chenwei Wu & Zhaochen Yu, with wonderful collaborators across PhAI Labs, CUHK, Stanford, MIT, Harvard Medical School, Oxford, Princeton, PKU and more.
Grateful to @james_y_zou @Jiaru_Zou @Charles_Y_Wu @zhenfei_yin_yzf 🙏
Ramakanth Kavuluru retweeted
1/ RL for improving task success has made LLMs remarkably capable. But what happens when the model fails? Turns out, training for graceful failure can be surprisingly tricky!
Very excited to share our new paper: Training Language Models to Fail Gracefully
🧵
Ramakanth Kavuluru retweeted
Researchers tend to have their best, most innovative ideas early in their careers. But for decades, researchers have had to wait longer and longer to test their own ideas.
Thrilled by the progress we’ve made so far on this front, including the huge milestone of setting a new record for grants supporting early-stage investigators.
ALT A graphic highlighting "A Record Year!" with 1648 early-stage investigators receiving NIH support through R01-equivalent extramural research grant awards. The background shows a blurred image of people working in a laboratory, with the U.S. Department of Health and Human Services and NIH logos at the bottom.
Ramakanth Kavuluru retweeted
🧵[1/N] Are large language models more creative than humans at combining ideas? 🤔
Combinatorial creativity involves combining familiar ideas in unfamiliar ways, and is central to scientific discovery and technological innovation.
In fact, breakthrough scientific work is often associated with more distant combinations of methods, techniques, and concepts.
Here, we introduce Kombine, a new benchmark of combinatorial creativity for LLMs.
TLDR:
- LLMs are more combinatorially creative than the average human.
- However, conceptual blending is the hardest task for LLMs, and even frontier models struggle.
- LLMs also tend to homogenize, inventing concepts with the same properties across separate models.
- Moreover, humans and LLMs combine in distinct regions of semantic space.
- As a result, we anticipate greater conceptual diversity can be unlocked through human-AI collaboration. 🗣️
Thread below🧵⬇️
Ramakanth Kavuluru retweeted
WildChat, LMSYS, ShareChat… each human-LLM chat dataset aims to capture "real-world" AI use. But do they paint the same picture?
Not quite. From user messages alone, a classifier can name a chat's source dataset, and user models trained on them inherit these signatures 🧵👇
Ramakanth Kavuluru retweeted
What if,
for any scientific claim,
you can search 100M+ papers to find the strongest evidence for and against it
⚡️in 1 second?
That’s now possible with Evidence API! 🚀
paperclip.gxl.ai/evidence-ap…
For example, does drinking coffee reduce diabetes risk?
Ramakanth Kavuluru retweeted
We’re open-sourcing pplx-embed-v2-late, multi-vector embeddings for text and images, 9B and 0.6B, in one shared embedding space. You can use these to index multimodal data with 9B, and query on device with 0.6B. This also enables you to search over PDF pages with no OCR. And scores 92.4% on MADQA, 64% on BrowseComp+. Weights available on @huggingface now.
We're releasing pplx-embed-v2-late, two late-interaction embedding models that retrieve text, images, and pages with a shared embedding space for cross-model querying.
Both models achieve frontier performance and are publicly available on Hugging Face.
perplexity.ai/hub/blog/multi…
Ramakanth Kavuluru retweeted
At #COLM2026 tomorrow!
If swapping a demographic attribute flips an LLM's answer, is that bias? Compared to what? A paraphrase often does too.
Come chat about baselines for counterfactual prompting:
📍 Wed 10/7, 11am–1pm, Imperial #41
w/ @mosh_levy @yoavgo @byron_c_wallace
1/ (New paper!)
If swapping the gender in an input prompt makes the AI model give a different answer it means that it has to have a gender bias, right? Wrong.
🧵on counterfactual prompting for LLM evals:
Paper: arxiv.org/abs/2605.01048
Ramakanth Kavuluru retweeted
1/ As LLMs are integrated into our health systems, how can we efficiently evaluate whether evolving systems can reliably retrieve the information clinicians need?
Excited to share our new preprint: A Living Benchmark for Information Retrieval from Electronic Health Records.
Ramakanth Kavuluru retweeted
We built TrialPanorama: 1.6M clinical trials from 15 global registries, linked to ontologies and literature. Now in TrialMind for every user.
An 8B model trained on it beat 70B LLMs on all 8 clinical research tasks.
Demo: youtu.be/oNMdJFgn9ZY
Paper: arxiv.org/abs/2505.16097
Ramakanth Kavuluru retweeted
We’re open sourcing our state-of-the-art contextual embedding models, which perform best in turbopuffer’s context-bench.
We built a new way to train contextual embedding models, which encode each chunk of a document with the whole document in view.
pplx-embed-v2-context-9b-preview sets a new state of the art on ConTEB and @turbopuffer's new, privately held context-bench.
perplexity.ai/hub/blog/conte…
Ramakanth Kavuluru retweeted
How much is an AI token worth? We fit scaling laws to find out!
In August 2026, Pangram labels 31% of FineWeb-filtered web tokens as AI-generated, up from 10% in June 2024.
We pretrained 800 LMs (19.9M–973M params) on mixes of human and AI web text to measure what it does to pretraining 🧵
Ramakanth Kavuluru retweeted
Myth: early stage investigators (ESIs) are more likely to receive NIH funding by pursuing a smaller award such as an R21 over an R01-equivalent. ❌
The data reinforce an important point for ESIs: a smaller or shorter award does not necessarily mean an easier award. Among those who applied, ESIs had relatively strong funding rates for R01-equivalent awards, while funding rates for R21s were consistently lower than for R01-equivalents. 📈
bit.ly/46RKJta