Applied AI @SnorkelAI, Previously - @MistralAI , @llama_index (LlamaIndex) Building data and evals for frontier labs.

SF
Joined December 2010
AutoResearch experiment: I went into a GPU Mode kernel competition with very little CUDA background and engineered a research loop around coding agents: goal, rules, experiment memory, verifier, and compute. Finished 5th overall. Give it a read.
30
36
9
357
44,380
Ravi Theja retweeted
Really enjoyed Terry Tao's talk at @SAIRfoundation 's Science x AI Summit. Particularly liked this thoughtful categorization of the old and new worlds of math (pre and post-AI), with the degree of human understanding being the key variable to realizing the benefits of AI and avoiding the pitfalls. "Misaligned math" is also a nice way to term the possible bad case.
1
7
1
48
1,514
Ravi Theja retweeted
Introducing Chantless: Chapter 1 — Apprentice A powerful tool makes you free, allows you to express yourself, and learns and grows with you. This is what we're building with Chantless. A technology that lets you shape and evolve models alongside you.
31
26
6
159
9,056
Ravi Theja retweeted
Excited to see this incredible progress and leadership, especially for the US/Western open frontier! Congrats @reflection_ai !! 🚀
Introducing Beam: a highly efficient agentic open model with 501B total parameters and 23B active. - Frontier reasoning efficiency - Advances the Western open frontier on coding & agentic tasks - Trained end-to-end from scratch Full weights release this month. Learn more about Beam: reflection.ai/beam
2
4
1
41
2,192
Ravi Theja retweeted
Introducing Beam: a highly efficient agentic open model with 501B total parameters and 23B active. - Frontier reasoning efficiency - Advances the Western open frontier on coding & agentic tasks - Trained end-to-end from scratch Full weights release this month. Learn more about Beam: reflection.ai/beam
565
1,106
755
8,605
2,504,996
Ravi Theja retweeted
Today we’re introducing Extract v2.5 - a series of frontier agents tuned for document extraction. The agents (cost-effective, agentic, agentic plus) are tuned for value accuracy and grounding. Our extraction agents outperform Opus 5.5 and GPT-6 Sol while being 30%-4x cheaper. We’ve made massive improvements on complex extraction over * long lists (86.1% -> 95.5% on our agentic tier) * records spanning pages (85.5% -> 96.5%) * scanned forms (90.9% -> 95.7%) We’ve also launched the following features: ✅ Advanced citations: we locate bounding boxes for all supporting values for an inferred field, even if there's not an exact match. ✅ Structural Reasoning: we tailor document extraction algorithms depending on the type, layout, and information Our extraction agents are SOTA in price-performance on ExtractBench, across a wide range of cost points. We are the best tool for document extraction across documents of any complexity. Blog: llamaindex.ai/blog/introduci… All of these are available on LlamaParse: cloud.llamaindex.ai/ . Come check it out!
42
35
6
291
52,429
Ravi Theja retweeted
⏱️⚙️Introducing *AutoBenchmark* 📊🏁 - Creating benchmarks automatically - Benchmarking benchmark creation - Studying the role & impact of humans in the loop Blog post: facebookresearch.github.io/R… Key takeaways: 1) We find human-agent collaboration gives big wins over agents alone - fine-grained feedback in ideation stage crucial - autobench can be used to measure this in the future with stronger agents 2) We show that it's possible to make *autoresearch benchmarks* for AI research using this recipe – full recursive improvement loop! 3) Important Ingredients: Autobenchmark creation works best with feedback from two sources: benchmark solvers + external verifiers (human+AI). 🧵1/5
1
65
20
534
63,800
Ravi Theja retweeted
Strongly agree here. And a few quick thoughts: - (1) Basic research will increasingly be about the empirical science of studying and understanding, not just improving model performance. As AI models are increasingly *grown* as much as they are engineered, and as the complexity scale increasingly leads to emergent properties over discretely engineered ones, a lot of basic research is going to look more like classical empirical science - i.e. doing experiments, exploring, understanding, modeling, etc. (One type of experiment: a benchmark, if done correctly; that's what we are attempting to fund and support through @SnorkelAI Open Benchmarks Grants!) - (2) Basic research can prioritize *global optimization* (e.g. trying new architectures / algorithms / paradigms), which will always involve sacrificing local optimality at first. These first, less optimal steps - which could take months, or years - are hard for *any* commercially-funded organization to do fully (especially once basic economic gravity returns in some areas of the space). This is the gap that basic research institutions have always- and will always- need to fill! Also: these global optimization explorations do not always need 10K+ GPUs!! In fact, many will be selected specifically to explore new paradigms/approaches that are more compute efficient. - (3) Basic research has many other beneficial externalities, e.g. supporting training/teaching, open dissemination of knowledge, etc.
One of the worst forms of brainrot AI has cultivated is pessimism about basic research, ie the idea that important work can only happen inside a frontier/neo lab and only with 10k+ GPUs, so the rest should not even bother. What a bleak way to think about science. And it's false.
6
52
6,356
India has always idolised actors and cricketers. The last few years, entrepreneurs and founders have also gotten mobbed at the airports. How I wish we started celebrating scientists as much as we celebrate actors, cricketers, and founders. On that note, meet the Temple scientists behind Brain Flow: Dr. Divya Gulati (IISc), Dr. Sanchit Gupta (IISc), Dr. Yukti Chopra (SISSA Trieste), and Nitish Kumar (IIT Bombay). This team led, built, and validated the first continuous, wearable measurement of blood flow to the brain. Your brain uses 20% of your blood supply. It controls your hormones, sleep, immunity, and repair. That supply drops every year as you age, and you have never measured it. Even hospitals can only capture a single snapshot. Measuring and maintaining brain flow could be one of the most important health and longevity interventions of our time. At Temple, we have some of the best minds in the country, working on the one problem that sits upstream of every other: extending healthy human life. I am proud of our team, and looking forward to a lot of firsts and breakthroughs in the future.
Does @temple really monitor blood flow to the brain, or just the scalp? In our second validation study, we tested Temple BrainFlow against time-domain NIRS, one of the few technologies that can separate brain blood volume changes from scalp blood volume changes. We had 60 participants go through three physiological challenges: head-down tilt, stand-to-squat, stand-to-supine. Across all three challenges, Temple BrainFlow tracked brain blood volume at r=0.84-0.91, significantly more than scalp blood volume. The most significant result came from head-down tilt, where Temple's correlation with the brain was 0.91, and scalp correlation was just 0.49. Key takeaway: Temple tracks the brain, much more than the scalp. Why this matters: Cerebral Blood Flow responds to almost everything longevity-oriented people already do. It rises with aerobic exercise, and VO2max correlates directly with it. It rises with cold exposure. It rises with meditation. Better sleep sustains it. It drops when you sit for too long. It drops when you are dehydrated. It is one of the most responsive and integrative biomarkers of how well your body is actually running, and aging. Until now, you could only measure it in a clinical setting, in a single snapshot. Temple measures it continuously, in real time, while you live your life. And Temple is, as of today, the only wearable in the world with validations against brain blood volume, and not just scalp. This study was guided by our Scientific Advisor, Prof. David Boas, Director of Boston University's Neurophotonics Center, and the mind behind several new, high-impact, biomedical optical technologies in neuroscience, including NIRS. This is the second modality Temple BrainFlow is now validated against, after Transcranial Doppler. More coming soon. Link: biorxiv.org/content/10.64898… The paper is posted as a preprint and submitted for peer review. We encourage researchers to replicate this. We'll happily loan Temples to any qualified researcher who wants to pressure-test any of our metrics independently. Cc @agingroy
100
243
23
2,597
171,337
Ravi Theja retweeted
🥳 Excited to share that @lossfunk got 4 papers into NeurIPS main conference. Please join me in congratulating the authors! Aman @arcaman07, ⁠Sushrut @martisamuser, ⁠Abhinav @MajorTimbWlf21, Pranav - @Avg_sapient First authors are all undergrads :)
59
70
10
1,484
74,888
Ravi Theja retweeted
I'm excited to announce @SnorkelAI's $350M Series E at $3.5B, led by @insightpartners and @S32_VC. We've grown 18x+ in the last 12 months since launching our Data-as-a-Service offering, passing $375M ARRR this week. As AI advances to superhuman capabilities, AI data & environment development must advance with it - and basic staffing and crowdsourcing approaches are not enough. AI progress now requires deep research and technology work that combines human expertise with specialized AI in compounding ways. @SnorkelAI is building the RSI data engine and frontier data lab for this next phase. We're honored to have the support of existing investors Addition, @lightspeedvp, @GreylockVC, @GVteam, P7, Factory, @WellsFargo, Walden Catalyst Ventures, and new investors @ThirdPointLLC, @MarchCPs, @BlumbergCapital, @AllegisCapital, @Frontlinevc, and @standard_vc. – @SnorkelAI started as a research project a decade ago at @StanfordAILab. Our thesis was simple: AI progress would become increasingly data-centric – and therefore data development should be studied as a true research and technology problem, not just a staffing and crowdsourcing one. Today, as AI capabilities verge on superhuman, building the data and environments to safely measure and train AI is becoming too hard for even the smartest human experts to do alone. Only humans and AI agents, collaborating together in compounding ways, can meet the accelerating needs of the frontier, and keep humans in the driver’s seat of AI progress for decades to come. At @SnorkelAI, we are building the data lab to define the shape of this new “Data 2.0” frontier, and the new paradigms of human-computer interaction needed to advance it. Our key focus is building the RSI engine for data, where specialized AI models accelerate and improve human expert output, and in turn, scaled human supervision is used to continuously evaluate and improve these models – creating a powerful compounding loop to keep pace with an accelerating RSI frontier. With this round of funding, we are also doubling down on our commitments to support data development for open benchmarking and evaluation (more news here soon!); an increasingly diverse ecosystem of general and specialized intelligence; and a path to safe, well-aligned AI built on robust training and evaluation data. Data development will guide and drive the next stages of AI – and must do so in a human-centric, AI accelerated, open, diverse, and safe way. We are excited to support this mission in the next decade of research ahead at @SnorkelAI. More thoughts here: snorkel.ai/blog/data-2-0-and…
73
117
51
417
91,151
We thought AI needed good computer-use models to play games. Turns out, all it needed was Jev.
jev is insane 🤯 Here is Jev playing subway surfers at super human speed, and also playing 50 games at once. cost less than a cent to do this run. Jev does not replace llms like astra or fable, but opens up an entirely new world of capabilities.
1
1
15
2,072
Interesting that 9/15 reviewed benchmarks are reported as "Flawed"
Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review.
5
836
Ravi Theja retweeted
i'm flattered by the shoutout, but there's no free lunch for document parsing Cohere parse starts at $1.50 / 1k pages, which is indeed on the cheaper end of doc OCR solutions, but it scores ~50% when averaged across all 5 ParseBench dimensions: * It lacks high-quality visual grounding, doing coarse segmentation for tables and images, but lacking finer-grained bboxes at the word/cell/line-level. this is quite important for any sensitive AI application that requires precise citations back to the source. * It lacks general visual / chart parsing capabilities, which are prevalent across finance, manufacturing, healthcare, and more * to its credit, it does a decent job over tables (87%) compared to other comparable solutions at the price point our cost-effective mode in LlamaParse starts at 0.375c / page and is more consistently strong across all dimensions. also there's various ways to bring it down with volume discounts if you're interested in learning more about various document parsing modes and comparing tradeoffs in price/accuracy, come talk to us! llamaindex.ai/contact LlamaParse: cloud.llamaindex.ai/
You're paying too much to parse your documents. Cohere Parse 5 fixes that.
22
11
2
122
25,927
Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3 Giang Son Nguyen, Nhi Ngoc-Yen Nguyen, Wray Buntine, Dung D. Le arxiv.org/abs/2609.04808 [𝚌𝚜.𝙲𝙻 𝚌𝚜.𝙰𝙸] 💬Accepted to BlackboxNLP 2026 Special Track
1
114
Building games with GPT-6 Astra is so much fun! Built a racing game set on Bengaluru’s ORR, from Bellandur to Marathahalli. Tried to capture the familiar chaos: traffic congestion, potholes (h/t @HarveenChadha), Swiggy, Zomato, and Blinkit delivery riders, and metro construction. And for some Road Rash nostalgia, you can swing a rod at vehicles alongside you to make room 😂 (h/t @ptkbhv). Give it a try: orr-rush-bengaluru.ravitheja…
5
4
22
1,459
Ravi Theja retweeted
You can just do things.
53
387
51
3,052
135,634
Ravi Theja retweeted
As engineering, product, design, DS, etc. melt into a new kind of role, I was reflecting on what roles might look like in the future. For example, when I look at the Claude Code team I see what I think is five archetypes: 1. Prototyper: comes up with brand new ideas; churns out many ideas, most of which don't ship 2. Builder: quickly turns a prototype/idea into production-grade product/infra 3. Sweeper: cleans up the UI, simplifies the code and system, unships, optimizes performance 4. Grower: takes a product that has been built and iterates on it to improve Product-Market Fit 5. Maintainer: owns a mature system to make it secure, reliable, fast, and efficient as it scales Many people span across 2 roles, and sometimes 3 roles. I also notice that these roles are not really tied to job function -- eg. across Anthropic, some designers match category 1, some 2, some 3; same for engineers, PM, DS. A healthy team needs a mix of these, depending on the product: - A product that is new and pre-PMF needs people that are strong at 1+2+3 - A product that is growing and has found PMF needs 2+3+4 and some 5 - A product that has strong PMF needs 3+4+5 and some 2 Maybe product roles of the future will look more like this, and less like the domain-specific roles of today?
883
2,450
688
20,099
3,157,615
Ravi Theja retweeted
We are releasing a fully reproducible early preprint of "Prism: Unlocking Language Model Capability Extraction". A trained language model knows many things at once, but deployment usually asks for one behavior at a time. Enterprise scenarios often have few products, workflows, features, or use-cases matter disproportionately. Prism asks and answers a simple question - "Is it possible to isolate and deploy only capabilities that are driven by Pareto principle and cut down costs by a huge margin while preserving most of the performance?" This paper discusses a novel approach to efficiency, understanding model behavior and opens up capability extraction.
21
38
6
221
24,755