@joinHandshake

Building the future workforce of the AI economy 🤝

 United States
joinhandshake.com/
Joined March 2014
  • Tweets3.2K
  • Following351
  • Followers12K
913Media
Introducing Handshake AI—the most ambitious chapter in our story. We leverage the scale of the largest early career network to source, train, and manage domain experts who test and challenge frontier models to failure for the top AI labs.
44
23
8
214
359,070
ChatGPT for Education analyzed thousands of professional chats with AI. After removing routine tasks like email and calendaring, a little over 40% of what remained was people asking questions and doing work outside their role. Roles are blurring. That makes skills-based and project-based hiring more important than ever: people are stretching beyond their job descriptions, and employers increasingly want that flexibility too.
1
1
1
5
460
Handshake works with thousands of working scientists to measure where frontier AI falls short on real research tasks, and to build the data that closes the gap.
Jonas Mueller
@jomulrOct 16
We tested Gemini 4, Muse Wonder, GPT-6.1 Sol, and Claude Opus/Sonnet 5.5 on images scientists interpret at work. None beats GPT-6 Astra, which fails on 50% of the 161 images. How can lab-in-the-loop AI cure disease if models can't interpret the routine experimental results?
2
572
Congratulations to @reflection_ai on Beam, released today with a full safety report. Handshake is named in the report as a source for the conversations Beam was tested against, and among the partners who tested the model before release. Safety testing is expert work. Before a model ships it has to face the requests that matter: self-harm, child safety, cyber offense, fraud. The hard cases are not the obvious ones. A nurse or a researcher can ask something that sounds dangerous and isn’t. Someone has to write that case and decide where the safe answer ends. Our people wrote those cases for Beam and judged every turn. Making AI safe takes the same expert judgment as making it useful, and we do both. Report: reflection.ai/beam-safety-re…

Beam Safety Technical Report — Reflection

Make intelligence open and accessible to all

reflection.ai
Reflection
@reflection_aiOct 16
Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners. We're releasing: - Weights, tokenizer, and reference implementation - BF16, FP8 and NVFP4 weights and a DFlash2 drafter - The open source harness we built and trained Beam on: Mirror CLI Weights and model card on Hugging Face: huggingface.co/reflection
1
2
13
1,937
Handshake AI is advancing open models with expert data
NVIDIA AI
@NVIDIAAIOct 15
Meet NVIDIA Nemotron 3.5 Super VL Our new open model combines reasoning and tool use with text, image, document, and video understanding for long-running AI agents. 120B total parameters with 12B active 🧵
1
2
14
987
We collaborated with @NVIDIA AI to post-train Nemotron 3.5 Super VL on over 2,000 data points from experts on top enterprise R&D teams, doubling its accuracy on life sciences artifact interpretation. @joinHandshake is building accurate, efficient models to let researchers accelerate their work and focus on the scientific decisions that drive discovery. Read the research report: joinhandshake.com/research/a…
5
7
7
44
5,709
Fellows: your dashboard is changing so you can see what you earned, how it was calculated, and when it lands. You’ll see base pay, incentives, and adjustments by project, plus a reason for every change and payment status. Rolling out across projects over the next few weeks. Full update here: joinhandshake.com/blog/our-t…

Handshake AI pay transparency for Fellows

Handshake AI is making Fellow pay clearer with a new payments dashboard, adjustment explanations, time tracking, and an easier dispute process.

joinhandshake.com
1
2
8
2,813
What a week at @COLM_conf in SF! We loved meeting the research community, hearing about the work people are doing, and sharing more about our ATLAS benchmark collection at the Handshake Al booth. Also, a huge thank you to our friends at @Cerebras for cohosting Cafe Compute with us. It was a blast! Explore the full set of ATLAS benchmarks → joinhandshake.com/research/b…
1
2
17
1,145
“The model’s capabilities are pretty spiky.” — Jonas Mueller
Jonas Mueller
@jomulrOct 8
The ultimate benchmark for new models is how useful professionals find them to get stuff done. Explaining benchmaxxing on The Information TV this morning, and why there’s no substitute for actual feedback from scientists/doctors using the model to solve their problems.
2
1
9
916
The best GRE prep used to be a budget question. This benchmark says it doesn't have to be.
Nishant Ranka
@nishantXrankaOct 8
Replying to @joinHandshake
@joinHandshake ran a controlled study with 2800+ students comparing the impact of AI with human tutor. Two interesting findings: (1) AI and human produced similar learning gains, and (2) AI is 70x (Opus) to ~900x (Gemma) cheaper. See: ATLAS-Education joinhandshake.com/research/b…

ATLAS-Education: AI matches expert human GRE tutors

A benchmark of 2,383 students shows AI tutoring matches expert human tutors on GRE learning gains (p=.015)—one AI tutor at 918x lower cost.

joinhandshake.com
1
3
1
12
1,376
Agent benchmarks should be grounded in how real people actually interact. This often requires user simulation, but current models often lack fidelity. After months of development, our research team has published CUE, a simulator that outperforms state-of-the-art approaches on fidelity and its ability to reproduce outcomes with real users. We’ve released the research, code, and trained embedding model openly for researchers to build better multi-turn benchmarks!
Anjali Kantharuban@anjali_rubanOct 6
⚠️ User simulators are increasingly used to evaluate AI agents in multi-turn interaction. But we find that looking like real users ≠ reproducing real evaluation outcomes. 📢 We introduce Calibrated User Embeddings (CUE) to bridge this gap across LLM simulators. 🧵 1/8
9
1,116
Handshake retweeted
Jonas Mueller
@jomulrOct 6
A user simulator isn’t useful just because it looks like a real user. AI agents should also succeed/fail with simulated users the same way they do with real ones. To fix this, we open-sourced CUE, a new simulator to help you build more ecologically valid multi-turn benchmarks.
Anjali Kantharuban@anjali_rubanOct 6
⚠️ User simulators are increasingly used to evaluate AI agents in multi-turn interaction. But we find that looking like real users ≠ reproducing real evaluation outcomes. 📢 We introduce Calibrated User Embeddings (CUE) to bridge this gap across LLM simulators. 🧵 1/8
3
3
1
37
2,169
As many industries adopt AI, we sat down with Douglas Melsheimer, Co-Head of Emerging Tech Investment Banking at @jpmorgan, to discuss how it’s changing the way we work.
1
2
1
13
1,350
We also asked how he thinks about AI’s role in banking today, and how he’s coaching his team in a rapidly changing environment.
1
1
426
“It’s going to take an insane amount of people.” At our AI Skills Studio launch, @Box CEO @levie joined Handshake’s @jonstull to talk about the new jobs AI is creating, and the engineering skills businesses need to put it to work.
2
9
29
3,035
Great to see @googlegemma highlight StudentBench from the Handshake AI research team. We’re excited to help build smarter AI that helps people think better too!
Google Gemma
@googlegemmaOct 1
Replying to @googlegemma
Credit to @cgnorthcutt, @joinHandshake, & co-authors 🧵 Thread: xitter.cf/cgnorthcutt/status/210… 📄 Paper: arxiv.org/abs/2609.28470 📊 Data: studentbench.org
2
9
1,449
Handshake retweeted
Google Gemma
@googlegemmaOct 1
Credit to @cgnorthcutt, @joinHandshake, & co-authors 🧵 Thread: xitter.cf/cgnorthcutt/status/210… 📄 Paper: arxiv.org/abs/2609.28470 📊 Data: studentbench.org
Curtis G. Northcutt
@cgnorthcuttSep 24
Today we establish that AI is as good as expert human tutors for immediate GRE learning gains (p=.015, n=2,383 students, study conducted July-Sep 2026). One hour with an expert GRE tutor: $75. One equivalent hour with an AI tutor: 7 cents . 918x cheaper, equivalent learning (p=.044, n = 140). In 5 of 7 academic topics, the top AI tutor beat the expert human tutors on average. How we measured it: pre-test -> 1 hour with (AI or human or no) tutor -> post-test. There are no LLM judges. Learning gains are post-pre. Everything is real students. Why we did this: > 1 trillion USD is being spent per year on machines making machines better. StudentBench shows us how to shift these resources so machines also make humans better. Why I did this: I grew up in rural Kentucky. My dad, grandad, and great grandad were all mailmen. Getting into a good college changed my life. Now we have done a rigorous study to help AI labs evaluate LLMs and help show how bright AI can be for students of all incomes. I believe in good science that leads to the democratization of opportunity. The research paper behind StudentBench: arxiv.org/abs/2609.28470 Data, code, details: in this thread If you're wondering what I'm selling, sorry to disappoint: we released all study data for free on Hugging Face, and made studentbench.org free to use. Enjoy. lots more in the video / thread!!
2
4
2
32
8,006
We're teaming up with @cerebras for Cafe Compute: Researcher Edition next Thursday in SF Excited to gather the best of the SF research community in one room. One attendee walks away with a NeurIPS All Access Pass + a free roundtrip flight to Sydney. Data + compute = AGI. Come talk about both. RSVP here → luma.com/cafecomputeresearch
4
14
3,387
Load more