
Applied AI Leader EMEA @AnthropicAI ex-@SnorkelAI & @datarobot. A French in London. AI/ML/Tech. Tweets reflect my own views.
London
Joined February 2010
- Tweets1.3K
- Following851
- Followers543
- Likes854
André Balleyguier retweeted
I'm excited to announce @SnorkelAI's $350M Series E at $3.5B, led by @insightpartners and @S32_VC.
We've grown 18x+ in the last 12 months since launching our Data-as-a-Service offering, passing $375M ARRR this week.
As AI advances to superhuman capabilities, AI data & environment development must advance with it - and basic staffing and crowdsourcing approaches are not enough.
AI progress now requires deep research and technology work that combines human expertise with specialized AI in compounding ways. @SnorkelAI is building the RSI data engine and frontier data lab for this next phase.
We're honored to have the support of existing investors Addition, @lightspeedvp, @GreylockVC, @GVteam, P7, Factory, @WellsFargo, Walden Catalyst Ventures, and new investors @ThirdPointLLC, @MarchCPs, @BlumbergCapital, @AllegisCapital, @Frontlinevc, and @standard_vc.
–
@SnorkelAI started as a research project a decade ago at @StanfordAILab.
Our thesis was simple: AI progress would become increasingly data-centric – and therefore data development should be studied as a true research and technology problem, not just a staffing and crowdsourcing one.
Today, as AI capabilities verge on superhuman, building the data and environments to safely measure and train AI is becoming too hard for even the smartest human experts to do alone.
Only humans and AI agents, collaborating together in compounding ways, can meet the accelerating needs of the frontier, and keep humans in the driver’s seat of AI progress for decades to come.
At @SnorkelAI, we are building the data lab to define the shape of this new “Data 2.0” frontier, and the new paradigms of human-computer interaction needed to advance it.
Our key focus is building the RSI engine for data, where specialized AI models accelerate and improve human expert output, and in turn, scaled human supervision is used to continuously evaluate and improve these models – creating a powerful compounding loop to keep pace with an accelerating RSI frontier.
With this round of funding, we are also doubling down on our commitments to support data development for open benchmarking and evaluation (more news here soon!); an increasingly diverse ecosystem of general and specialized intelligence; and a path to safe, well-aligned AI built on robust training and evaluation data.
Data development will guide and drive the next stages of AI – and must do so in a human-centric, AI accelerated, open, diverse, and safe way. We are excited to support this mission in the next decade of research ahead at @SnorkelAI.
More thoughts here: snorkel.ai/blog/data-2-0-and…
André Balleyguier retweeted
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
André Balleyguier retweeted
Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software.
It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans.
anthropic.com/glasswing
André Balleyguier retweeted
It will significantly increase my opinion of @Anthropic if they do not back down, and honorably eat the consequences.
(For those who are not aware, so far they have been maintaining the two red lines of "no fully autonomous weapons" and "no mass surveillance of Americans". Actually a very conservative and limited posture, it's not even anti-military.
IMO fully autonomous weapons and mass privacy violation are two things we all want less of, so in my ideal world anyone working on those things gets access to the same open-weights LLMs as everyone else, and exactly nothing on top of that. Of course we won't get anywhere close to that world, but if we get even 10% closer to that world that's good, and if we get 10% further that's bad)
CC @DarioAmodei
firefly.social/post/bsky/pv7…
André Balleyguier retweeted
Excited to share an example of the many projects we're driving @SnorkelAI around enterprise-specific environments and benchmarks - including detail on:
- Domain-specific, enterprise env & tool development
- Persona simulation for multi-turn eval
- Nuanced rubrics
& more!
We’re heading to #AAAI2026!
Our accepted paper “Benchmarking Agents in Insurance Underwriting Environments” will be presented Jan 26 at the AAAI workshop.
Stop by to learn more—@amanda_dsouza will be there to chat!
Learn more about our benchmark: snorkel.ai/blog/building-the…
André Balleyguier retweeted
Excited to share a preview of @SnorkelAI 's new Agentic Coding benchmark - testing models on realistic, multi-step software engineering tasks in fully sandboxed execution environments across a calibrated range of task domains and difficulties, inspired by our work with the @terminalbench team!
With a top pass@5 score of 58% (Opus 4.5) - this new benchmark challenges the notion running wild on X right now that LLMs have "solved" software engineering.
And, with both unit tests and both final-output and trajectory-level rubrics, it's already giving us & partners insights into where coding agents fail. Excited to share more here shortly!
Link to benchmark & release post in 🧵👇
André Balleyguier retweeted
Don't say: "vLLM is fastest" or "Ollama is easiest."
Wrong framing.
The real answer isn't about features - it's about matching serving philosophy to your constraints.
Local prototype vs. production scale vs. complex workflows = completely different frameworks.
André Balleyguier retweeted
I rejected a job offer yesterday.
Not because of the salary.
Not because of the tech stack.
Not even because of the long hours they warned me about.
But because, when I asked how they evaluate their AI systems, the hiring manager said:
"We just ask it some questions and see if the answers sound right."
I stared at them for a moment and realized... They just described the biggest problem in AI today.
See, "sounds right" isn't a measurement.
It's a hope.
Here's what proper LLM evaluation actually looks like:
- Accuracy: Can it get factual questions right? (Not 80% of the time. Consistently.)
- Hallucination rate: How often does it make things up? (This should be near zero for critical applications.)
- Bias metrics: Does it treat all groups fairly? (Measured across demographics, not assumed.)
Real Evaluation Frameworks:
- BLEU scores for translation quality Perplexity for language modeling Human evaluation with inter-annotator agreement Adversarial testing (red teaming) Domain-specific benchmarks (legal, medical, financial)
The Process:
> Define success criteria BEFORE deployment
> Create diverse test sets (not just happy paths)
> Measure consistently across model versions
> Track performance over time (models drift)
Have humans validate edge cases
Why This Matters:
Before proper evals: "Our model is amazing!"
(based on cherry-picked examples)
After proper evals: "Our AI achieves 94.2% accuracy on domain X, with known failure modes Y and Z"
The difference? One builds trust. The other destroys it when reality hits.
The kicker: Most companies are still in the "sounds right" phase. They're deploying models evaluated by vibes, not metrics.
Just like you wouldn't join a team that deploys code without tests, you shouldn't join one that deploys AI without proper evaluation.
What's your experience with LLM evaluation?
Are we measuring what actually matters?
André Balleyguier retweeted
Tested gpt-oss-120b on @SnorkelAI 's agent benchmarks - performance seems split on real world tasks. It did well on multi-step Finance, but poorly on multi-turn Insurance. Think this could be a limitation of either domain knowledge or handling multi-turn scenarios.
André Balleyguier retweeted
Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with @SnorkelAI Data-as-a-Service, and to share our new leaderboard!
—
Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with @SnorkelAI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases!
Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale.
This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components:
- (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains.
- (2) @SnorkelAI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D.
Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)!
We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview:
- SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning
- SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use
- SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning
André Balleyguier retweeted
It’s been amazing to watch @ajratner build and grow @SnorkelAI from a neurips presentation and an open source repo to one of the highly valuable AI companies building impactful product features for data labeling. Very timely that they are bringing in that expertise to agentic workflows! If you still wondering why you need data in this world of high performing models, see the next tweet.
Agentic AI will transform every enterprise–but only if agents are trusted experts.
The key: Evaluation & tuning on specialized, expert data.
I’m excited to announce two new products to support this–@SnorkelAI Evaluate & Expert Data-as-a-Service–along w/ our $100M Series D!
---
Snorkel Evaluate is our new data-centric agentic AI evaluation platform for specialized, mission-critical enterprise settings where vibe checks and out-of-the-box metrics driven by simple LLM prompts are not enough.
Snorkel Expert Data-as-a-Service is our white glove service for expert-level AI datasets, powering frontier LLM developers in areas like expert knowledge, reasoning, agentic action and tool use, and more!
Both built on top of @SnorkelAI’s Data Development Platform, using our programmatic technology to drive higher-quality expert data, faster– for getting specialized AI to real production value.
If you’re building enterprise AI and want to partner around the key ingredient in AI today–the data–book a demo and let's talk! snorkel.ai/demo/
Finally, see thread for details on 🧵👇
- 📽️ A walkthrough of Snorkel Evaluate and Expert Data-as-a-Service on an agentic AI enterprise task
- 📅 An upcoming event on Enterprise Agentic AI with innovators from @Accenture @BNY @Comcast @Stanford @QBE & others
- 📊 An upcoming series of benchmark datasets and model artifact releases
👀 Want early access to the full agentic AI dataset? Retweet this post and we'll send you the link!
André Balleyguier retweeted
High quality data is key. Excited to work with Snorkel to improve our Marin models!
Agentic AI will transform every enterprise–but only if agents are trusted experts.
The key: Evaluation & tuning on specialized, expert data.
I’m excited to announce two new products to support this–@SnorkelAI Evaluate & Expert Data-as-a-Service–along w/ our $100M Series D!
---
Snorkel Evaluate is our new data-centric agentic AI evaluation platform for specialized, mission-critical enterprise settings where vibe checks and out-of-the-box metrics driven by simple LLM prompts are not enough.
Snorkel Expert Data-as-a-Service is our white glove service for expert-level AI datasets, powering frontier LLM developers in areas like expert knowledge, reasoning, agentic action and tool use, and more!
Both built on top of @SnorkelAI’s Data Development Platform, using our programmatic technology to drive higher-quality expert data, faster– for getting specialized AI to real production value.
If you’re building enterprise AI and want to partner around the key ingredient in AI today–the data–book a demo and let's talk! snorkel.ai/demo/
Finally, see thread for details on 🧵👇
- 📽️ A walkthrough of Snorkel Evaluate and Expert Data-as-a-Service on an agentic AI enterprise task
- 📅 An upcoming event on Enterprise Agentic AI with innovators from @Accenture @BNY @Comcast @Stanford @QBE & others
- 📊 An upcoming series of benchmark datasets and model artifact releases
👀 Want early access to the full agentic AI dataset? Retweet this post and we'll send you the link!
André Balleyguier retweeted
Agentic AI will transform every enterprise–but only if agents are trusted experts.
The key: Evaluation & tuning on specialized, expert data.
I’m excited to announce two new products to support this–@SnorkelAI Evaluate & Expert Data-as-a-Service–along w/ our $100M Series D!
---
Snorkel Evaluate is our new data-centric agentic AI evaluation platform for specialized, mission-critical enterprise settings where vibe checks and out-of-the-box metrics driven by simple LLM prompts are not enough.
Snorkel Expert Data-as-a-Service is our white glove service for expert-level AI datasets, powering frontier LLM developers in areas like expert knowledge, reasoning, agentic action and tool use, and more!
Both built on top of @SnorkelAI’s Data Development Platform, using our programmatic technology to drive higher-quality expert data, faster– for getting specialized AI to real production value.
If you’re building enterprise AI and want to partner around the key ingredient in AI today–the data–book a demo and let's talk! snorkel.ai/demo/
Finally, see thread for details on 🧵👇
- 📽️ A walkthrough of Snorkel Evaluate and Expert Data-as-a-Service on an agentic AI enterprise task
- 📅 An upcoming event on Enterprise Agentic AI with innovators from @Accenture @BNY @Comcast @Stanford @QBE & others
- 📊 An upcoming series of benchmark datasets and model artifact releases
👀 Want early access to the full agentic AI dataset? Retweet this post and we'll send you the link!
André Balleyguier retweeted
Incredibly excited about @SnorkelAI's R3 release - bringing programmatic data development to:
- LLM evaluation
- LLM prompting, fine-tuning, & alignment
- Advanced PDF extraction
and much more, all with best-in-class enterprise readiness! snorkel.ai/blog/snorkel-flow…
Details 🧵👇
André Balleyguier retweeted
Excited to share a new, free custom LLM evaluation offer! snorkel.ai/snorkel-custom-ev…
And a webinar + demo on the programmatic data development technology behind it youtube.com/watch?v=ZHjulqB-…
Why? Because current eval solutions are not specialized, fine-grained, or scalable enough👇🧵
André Balleyguier retweeted
Some thoughts on RAG:
- How to get it working via tuning & indexing
- How this is powered by the "fuel" of labeled data
- How this connects to our programmatic data dev platform @SnorkelAI!
+ a mini rant on how the AI world has forgotten about the field of info retrieval :)
This is a great topic to highlight. The tl/dr is that RAG systems and their underlying databases need to be tuned and indexed- just like with any other information retrieval system! And labeled data is key here.
As you can guess: @SnorkelAI we've been supporting an increasing number of RAG tuning use cases w/ our programmatic approach to data labeling, in two major categories:
- (1) RAG Tuning: Retrieval systems need to be taught how to find the right chunk/context for a specific domain/use case. Here labeled data (prompt/context pairs) are key, and can be used to tune the RAG system (i.e. embeddings and/or retrieval/re-ranker models).
- (2) RAG enrichment: Underlying context data can be pre-labeled/tagged to help better index for RAG, as @levie highlights above!
We've helped major customers (e.g. top-10 US banks) go from e.g. 25% -> 90%+ accuracy in RAG-based Q&A systems, in weeks or less. In many of these cases- it's the RAG system not the LLM that's the biggest error mode- and all about efficient labeled data to tune/index!
FWIW (mini weekend rant)...
- The fact that RAG systems need tuning has been known since the original RAG paper (arxiv.org/abs/2005.11401) where both the retrieval model and LLM were fine-tuned.
- Information Retrieval (IR) is an entire, decades-old field of study. Most of the principles still apply, and have not been overhauled just because we have more powerful embedding vectors... yet most of AI Twitter seems to be trying to re-invent this entire classical field vs. reading a 101 text book from a decade ago, which would be a *much* more productive start ... :)
André Balleyguier retweeted
Some thoughts on Alignment:
- It's just fine tuning (basically)!
- It's all about the labeled data
- Some recent work @SnorkelAI on what we call "enterprise alignment" (to distinguish from ✨superalignment✨)
+ a mini rant on what we need to focus on to actually make AI work :)
First, basic definitions:
- "Alignment": aligning AI models to specific preferences, objectives, standards, etc.
- "Superalignment": aligning future, smarter-than-human AGIs to our objectives, ethics, etc.
Second, a simple point: there have been a plethora of terms to describe some form of tuning an LLM with respect to a labeled dataset:
- "Fine tuning": Tuning on labeled training data (some association/connotation of non-generative tasks, e.g. classification etc?)
- "Instruction tuning": Tuning on prompt, response pairs.
- "Alignment": Tuning on some form of preference data (e.g. LLM A vs. B, yes/no, written feedback, etc.)
For a while, there was a lot of excitement around RL-based algorithms for alignment, leading to near synonymity between "alignment" and "RLHF". However:
- (A) Non-RL approaches like DPO et al have since seemed to overtake in popularity (and there's ample theoretical + empirical basis for this shift), leading to alignment algorithms that look just like standard, non-RL fine tuning algorithms
- (B) Conceptually, it's just tuning LLMs for a labeled dataset either way!
For the above reasons- @SnorkelAI we just prefer to call the above all "fine-tuning" for simplicity; and, believe that this (+ prompting) will all eventually converge and become lower-level details of LLM adaptation.
What will matter then? And what matters most now, as the algorithms and infrastructure for doing fine-tuning has already standardized and commoditized? The labeled data!
This is evinced by BigTech and LLM providers spending $ Bs on labeled data for fine-tuning and alignment, and organizations like OAI beginning to research more scalable approaches for future superalignment objectives, like weak-to-strong generalization (which we started working on 8+ years ago, arxiv.org/abs/1605.07723).
The problem for enterprises: the use cases, objectives, policies, regulations, tone of voice, etc. that they need their AI models to align with are *not* the same as the generic ones these LLM providers are tuning/aligning to!
We call the challenge of aligning to enterprises' specific objectives/policies enterprise alignment. It's difficult especially because private data & internal subject matter experts are usually required to do the data labeling (i.e. can't just be outsourced, even if there were $ Bs to spend!)
@SnorkelAI we've show that our programmatic approach to data labeling & development can solve this challenge, leading to custom alignment in a few hours of development- for ex., aligning a finance chatbot to be compliant with financial advisory policies (+20.7 pts. above baseline) in several hours.
Check out our work here: snorkel.ai/introducing-enter…. More details and content- both in our platform, Snorkel Flow, and OSS- coming soon!
As for the "mini rant" - mostly bait to read a long post on a niche technical topic :). However, many of us in AI have a strong view that:
- (A) Superhuman AGIs are not right around the corner.
- (B) Enterprises are struggling to deploy AI to impactful production settings- and the party's over for all of us (and all the positive impact that we *know* AI can have) if we don't deliver value ASAP.
- (C) Safe systems at scale are usually built through years/decades of incremental and collective engineering (e.g. think air travel), vs. secretive leaps forward.
That is: let's all learn to walk safely first- then we'll get to flying saucer seatbelts!
André Balleyguier retweeted
There's a new programming language in town - it's Mojo! I'm more than a little excited about it. It's Python, but with none of Python's problems.
You can write code as fast as C, and deploy small standalone applications like C.
My post is below, and a 🧵
fast.ai/posts/2023-05-03-moj…













