making AI work

Joined July 2016
The future of enterprise AI is multi-model, and enterprises deserve the freedom to choose. NEWS: We’re expanding that choice through our collaboration with @NVIDIA, post-training its open-weight Nemotron 3.5 Super VL for enterprise workflows. Here’s what it delivered for two of our customers: ✅ @Skydance: A complex finance workflow that reviews vendor credit memos, statements and invoices to find credits the company is owed. Post-trained Nemotron matched their current frontier model’s accuracy at 40x lower inference cost, with 5x faster response times. ✅ @EY_US: An M&A due diligence agent that finds and reconciles answers across hundreds of files in a virtual data room. Post-trained Nemotron matched EY’s current frontier model at about one-eighteenth of the inference cost, returning answers more than twice as fast. Put NVIDIA’s open-weight models to work for your business with Scale Post-Training. Thank you, @JensenHuang and the NVIDIA team, for the early access and collaboration. scale.com/partnership/nvidia
4
7
3
44
3,386
the coolest thing we see as the organizers is when events are over capacity. @scale_AI clearly resonated with folks last night!!
1
2
17
1,600
People increasingly turn to AI chatbots for mental health support. But are models actually helping when it matters most? We're releasing DistressBench, a new benchmark measuring how well AI models respond to mental health crises.
1
7
24
1,953
We’re introducing Humanity’s Sixth Sense, a new benchmark testing the intuitive visual reasoning people rely on every day. Across 522 open-ended tasks spanning images and video, HSS tests everything from spatial and causal reasoning to social understanding. The gap is significant. Humans score 93.1%, while the strongest model, GPT-6-astra, reaches 53.6%. The median model scores just 30.9%.
85
148
93
2,242
207,489
Much of the AI security debate is about frontier models. But a lot of people are building on open-weight models. We need to understand their capabilities and risks too. A real testing strategy covers every model, not just the frontier.
1
9
28
3,400
You can’t improve agents if you can’t measure where and why they fail. At #LATechWeek, we're hosting a hands-on evals workshop led by our Head of General Agents @Bckenstler to break down how we structure evals, what we measure, and how we use the results to improve AI systems. Apply here to join us: scl.ai/LATechWeek
2
6
1
19
2,078
An honor to meet with His Highness the Crown Prince of Kuwait to discuss how @Scale_AI can support Kuwait’s digital transformation and develop Kuwaiti AI talent.
ممثل حضرة صاحب السمو الشيخ مشعل الأحمد الجابر الصباح أمير البلاد سمو ولي العهد الشيخ صباح خالد الحمد الصباح يبحث مع الرئيس التنفيذي لشركة (Scale AI) العالمية فرانسيس ديسوزا آفاق التعاون المشترك في مجالات تطبيقات الذكاء الاصطناعي والبيانات الضخمة والتحول الرقمي - سموه استقبله والوفد المرافق له في مقر وفد دولة الكويت الدائم لدى الأمم المتحدة على هامش أعمال الدورة الـ81 للجمعية العامة في نيويورك - جرى استعراض سبل الاستفادة من حلول التكنولوجيا المتقدمة في دعم وتطوير القطاعات الحكومية والاقتصادية الحيوية في دولة الكويت وتأهيل الكوادر الوطنية وتنمية قدراتها - #الكويت #السعودية #فلسطين #القدس #غزه #اليمن #لبنان #السودان #سوريا #مصر #العراق #الأردن #الضفة_الغربية #حماس #ترمب #ترامب #أمريكا #تركيا #الرياض #ايران #ليبيا #صباح_الخير #يوم_الجمعة #ليلة_الجمعة #ساعة_استجابة #جمعة_مباركة #مساء_الخير #الخميس_الونيس #مضيق_هرمز #خليجي٢٧ #خليجي27 دول الخليج الدول العربيه الولايات المتحده الشرق الاوسط مجلس الامن الامم المتحدة الرئيس الفنزويلي مدريد صوت المطر صوت الرعد مطر الليل أبستين
8
6
35
4,925
The nation that can measure the AI frontier will be the one that steers it. Our future should not be steered by speculative warnings. We have to be driven by evidence. That was the clearest takeaway from this week's AI conversations at the UN General Assembly. Governments need to get serious about testing frontier AI, and nowhere is that more urgent than in Washington. Getting serious means three things: → Clear responsibility for testing across agencies → Access to models before they're deployed → Funding for the experts and tools to do the work Scale AI's cyber research shows what testing can reveal. We've seen AI agents carry out an attacker's instructions while still finishing the user's task, so the user never knew anything went wrong. We've also seen models refuse to help people defend their own systems. Both are failures. Real testing has to measure the harm AI can cause and the help it fails to give. Government shouldn't have to rely solely on AI developers to explain these failures. It needs its own testing capacity, with the expertise to ask hard questions and the tools to answer them. Scale has worked with the U.S. government for years to test the risks and capabilities of AI models, and we're accelerating that work in the months ahead. Here's what we think America should do next: scale.com/blog/steer-the-ai-…
3
10
37
1,936
This week I joined @RoyalFamily, govt ministers and leaders to discuss how AI can best serve the public good. AI can expand opportunity and solve some of our toughest challenges. It’s essential that we pursue that potential responsibly, in a way that benefits people broadly. Rigorous, independent testing - like we do at @Scale_AI - is an essential step to earning public trust. Governments and independent evaluators need to work together to deeply and continually assess AI's capabilities, opportunities, and risks. That allows us to make sound decisions about its development and use. Conversations like this one are a step in the right direction.
“I really am enormously grateful for the presence in this room of many of our greatest technological minds, and indeed, for the thoughtful stewardship demonstrated by those gathered here today as you consider the fundamental principles that should – and must – guide the development of Artificial Intelligence in the years ahead.” Today at Dumfries House, The King convened AI pioneers, Government representatives and thought leaders to discuss how AI can be developed and deployed in ways that benefit society.
2
4
30
3,980
We’re moving to a multi-model world and that’s a good thing. Enterprises will use multiple models, optimized for different scenarios. Thanks for having me on @andrewrsorkin and @BeckyQuick
Scale AI CEO @fdesouza sees the future of AI including more models, and particularly more narrowly-tailored technology. "We want models that are specific for different scenarios," he says. cnb.cx/4xBuTOX
1
6
43
7,007
How do you know if an agent is enterprise-ready? NEWS: Today Scale AI is introducing READY, our Reliable Enterprise Agent Deployment benchmarks, to answer that question. READY measures agents on reliability, the human oversight required to hit a defined reliability target, and the resulting cost, together. labs.scale.com/blog/ready
2
17
1
53
5,416
📣Call for contributions + co-authorship! RSI Bench is our ongoing effort to evaluate whether AI agents can develop the capabilities required to advance AI R&D. Got a frontier AI research problem on your mind? Submit it to RSI Bench, and set the standard the whole industry uses to measure progress! Contributors receive: → Co-authorship on the RSI Bench research paper → $2,000 per accepted task for the initial 50 tasks → Modal compute credits to build + iterate → Access to the RSI Bench research community Register below for full requirements.
Launching rsi-benchmark.com: The work of AI R&D has always belonged to humans. For the first time, though, it no longer seems certain that it always will. Recursive self-improvement is within a line of sight. It may still be far, but it is close enough that we should start measuring it.
13
30
11
297
73,074
Lots of talk over the past week about American data companies selling training data to Chinese AI labs, and why that leaves the US behind. Up front: Scale doesn't do this work, and we've turned down revenue over it. A lot of it comes down to "isn't this just labeling?" Fair question, and the answer is no, but I want to explain why… Frontier post-training data has very little in common with annotation at this point. It looks more like curriculum design for training models. You're deciding which problems a model should struggle with, how a reasoning trace should be structured, the difference between a right solution and a lucky one, and what to reward. Tasks, RL environments and Verifiers are that same judgment but in executable form. Which means what’s being sold is a set of research decisions, already made and already validated. That's also why it's not a small business for anyone involved, and it’s why the few companies like us in the space are doing well. We build frontier data pipelines from our research and build OTS datasets to match because buyers want ready-made inventory and capability gains. Not coincidentally, Chinese labs' biggest jumps are in exactly the domains where sellable environments exist: agentic coding, math, tool use, etc. A few things about this that I don't think got much attention: ➡️ Compute controls assume compute is the binding constraint. But RL post-training is bound by reward signal, not just FLOPs - a lab that buys verified environments doesn't waste compute on failed exploration, so it reaches the same capability on far fewer chips. That's a substitution the export regime doesn't account for. ➡️ Functionally, this hands labs distillation on easy mode, minus the legal exposure. The verifier confirms which teacher outputs are actually right, so you distill from verified outputs. And dense reward signals cut the compute RL burns on failed exploration. ➡️ The judgment being sold is American expert judgment. The people writing these rubrics, the domain PhDs and engineers and professionals, generally have no visibility into who the end buyer is. They think they're contributing to work they'd endorse. ➡️ As recently reported some of the same vendors hold US government contracts. That's not a hypothetical conflict, and it should be getting more scrutiny. We shouldn’t assume bad intent from anyone. This is a supply chain that grew faster than our ability to think about what the rules should be. If you're buying data, "what am I getting" is only half the diligence. Where else that pipeline goes is the other half, particularly from companies that talk publicly about American AI leadership.
2
8
72
4,341
American companies should not be selling data to Chinese AI labs. @scale_AI doesn’t do this work, and we have turned down revenue because of it. The companies that do are undermining American AI leadership and risking national security.
The AI arms race isn’t just being fought over Nvidia chips. Chinese labs are matching OpenAI and Anthropic by purchasing the exact same data from US vendors like @mercor and Surge AI (who also work with the US federal govt) forbes.com/sites/annatong/20…
13
27
6
219
231,245
Big News: I’m joining @scale_AI as CEO, starting August 10. Scale sits at a rare intersection, working with the top AI labs to push the frontier while helping enterprises and governments actually deploy AI they can verify and trust. I’m excited to lead a company with a mission to develop reliable AI systems for the most important decisions. More to come.
52
36
15
655
345,863
Proud to support this on behalf of @scale_AI Open models are critical to American AI leadership and to building reliable AI. Great to join so many across the industry in signing on.
3
13
1
88
7,949
We appreciate the community's feedback on SWE-Bench Pro. Much of it maps to changes already underway in v1.1, which we've been building for a while. Keeping evals current with frontier models is hard, and we're always iterating. SWE-Bench Pro Verified coming soon. 👀
8
19
6
181
40,462
Introducing the 6% Report. Everyone is talking about enterprise AI adoption, yet very few are producing real outcomes. Our new research shows only 6% of organizations have successfully deployed AI at scale and achieved measurable business value. Here's what we found they’re doing differently: scale.com/six-percent
4
11
1
66
17,422
Today we're releasing HiL-Dynamics, the first open-source tool that measures how production agents actually collaborate with humans under uncertainty. Not just whether they got the answer. Now you can measure exactly when your agent asks for help, when it makes assumptions, and when it'll confidently ship the wrong answer. Our findings 🧵
8
10
2
41
8,422
The humans stay. That’s the idea behind @scale_ai's new brand campaign. 10 years of building AI has taught us something: the most important decisions belong to humans. The AI that works in decisions of consequence keeps humans at the center. Going live in SF and NYC. Where to next? 👀
2
10
5
60
11,676