@Austin_Way
Account based inNorth America
About this account
- Account based in
- North America
- Connected via
- North America App Store
Account-level information from X, not a live location or the device used for a specific post.
17y/o @AlphaSchool Building the next generation of Ed-tech that will reach 1 billion kids.
Joined August 2023
- Tweets727
- Following186
- Followers6K
- Likes675
"AI tutoring encompasses lesson planning, text-based conversational tutoring, and practice problem generation." ... "Human tutors taught one-to-one in live video calls."
Could the AI results be summed up to students simply answering more practice questions?
Today we establish that AI is as good as expert human tutors for immediate GRE learning gains (p=.015, n=2,383 students, study conducted July-Sep 2026).
One hour with an expert GRE tutor: $75.
One equivalent hour with an AI tutor: 7 cents .
918x cheaper, equivalent learning (p=.044, n = 140).
In 5 of 7 academic topics, the top AI tutor beat the expert human tutors on average.
How we measured it: pre-test -> 1 hour with (AI or human or no) tutor -> post-test.
There are no LLM judges. Learning gains are post-pre. Everything is real students.
Why we did this:
> 1 trillion USD is being spent per year on machines making machines better. StudentBench shows us how to shift these resources so machines also make humans better.
Why I did this:
I grew up in rural Kentucky. My dad, grandad, and great grandad were all mailmen. Getting into a good college changed my life. Now we have done a rigorous study to help AI labs evaluate LLMs and help show how bright AI can be for students of all incomes.
I believe in good science that leads to the democratization of opportunity.
The research paper behind StudentBench: arxiv.org/abs/2609.28470
Data, code, details: in this thread
If you're wondering what I'm selling, sorry to disappoint: we released all study data for free on Hugging Face, and made studentbench.org free to use. Enjoy.
lots more in the video / thread!!
Austin Way retweeted
My school blocked the game I was playing so I made my own
This is Banana FPS (V.1)
Jev has accelerated the timeline for personalized education by years.
The biggest bottleneck in building personalized education with AI is validating the quality of the material it generates.
For the courses I’m building, generating lessons is only ~5% of the content cost.
The other ~95% is validation and improvement.
One of the biggest costs is checking whether a lesson gives a student too much information to process at once, overloading their working memory.
Running that analysis across all of our content with frontier models gets incredibly expensive.
At our current scale, even if Astra used half as many tokens as Jev, it would cost us ~$250,000.
With Jev: ~$2,000.
I built the tool we’re using for the working-memory check and made it available here:
github.com/AustinAWay/Workin…
I tried 17,000 times to get AI to write at a first-grade reading level.
It failed every time.
So I fine-tuned a Qwen model that scores near-perfectly on reading-level benchmarks.
huggingface.co/AustinWay/qwe…
“Have AI tutor you” is useless advice.
I tested 255 AI models, reviewing over 20,000 chats across 10 subjects.
AI’s ability to write at the requested grade level varies significantly by subject.
Despite the variation, the models never reliably hit the target reading level.
x.com/Austin_Way/status/2098…
Austin Way retweeted
I was reading this article, and Austin makes a great point.
There is a massive distinction between a chatbot teacher and a human teacher.
When a teacher talks, everything they say is said once.
There is no record of it afterward, and you subconsciously know that. So you put in more effort to understand and remember what they say.
We don’t treat chatbots the same way.
When you talk to Claude, you know every response will still be there whenever you need it.
That changes how much attention you give it.
We all know the feeling of getting a long answer to something that should have taken two sentences and just skimming through it.
And if you forget something, there is very little cost. You can always scroll back up.
That is the fundamental problem with AI tutors: the format itself incentivizes you to rely on the chat history instead of actually remembering what you learned.
Austin Way retweeted
15y/o. Built self stabilizing 6 legged robot.
Controlled entirely by an Iphone.
Made in under a 100 hours for less than $300.
💯
One of the most valuable things you can do is test with students. You learn things you'd never think of.
I've seen so many edtech products w/ content two grades above the student's reading level.
It's so hard to realize that's even a problem until you watch students use your product. (One of many examples.)
Genuinely one of my favorite edtech teams right now!
AI is bad at writing educational lesson content.
3 problems:
1. Reading level is inconsistent.
2. AI ignores working memory limits.
3. AI avoids Direct Instruction.
You can tell AI to do all of these things in a prompt, but it still won't do them reliably.
My team and I have been working on a pipeline that checks and fixes all three:
"Just use ChatGPT to learn it" is useless advice.
AI doesn't know what to teach you first.
In a new study, AI tutors had to teach simulated students an entire subject. GPT-5-mini got them to full mastery just 3% of the time.
(btw: picking the next topic at random averaged 28% mastery).
So we mapped all 454 concepts in AP Human Geography and how they fit together.
If used in AI, this likely would result in high mastery.
At 17, I pitched my school's founder to let me build the AP social science edtech. (AP = Advanced Placement.)
That was 2 months ago. Today the practice test results are in.
AP Human Geography (13 students): avg 5.00
AP US History (9 students): avg 5.00
AP World History (9 students): avg 5.00
A 5 is a perfect score. The national average on these exams is a 2.91. Only 14% of students ever score a 5.
In just 6 weeks of using the program, students' practice test scores jumped:
Student 1: 1 → 5
Student 2: 2 → 5
Student 3: 2 → 5
Every student got there through advanced, personalized edtech. ( @AlphaSchoolATX is the future of education )