Generative AI can harm learning. A field experiment published in PNAS randomized nearly 1,000 high-school students across three conditions during mathematics classes: - textbooks and notes only
- a standard GPT-4 interface
- a safeguarded GPT-4 tutor that provided hints rather than answers Students first completed practice problems with these resources. Immediately afterwards, they took a closed-book, closed-laptop exam on similar problems. The results are impressive. During practice, AI made students look much better: Standard GPT-4: +48%
Safeguarded GPT tutor: +127% But when the AI was removed, students who had used standard GPT-4 performed 17% worse than students who had never used AI. The apparent improvement concealed a deterioration in learning. The safeguarded tutor eliminated this negative effect, but produced no improvement in independent exam performance. The reason? Simple: Students using standard GPT-4 frequently asked for the answer and just copied the solution. They surrendered cognition. I’m very concerned, especially for children who will have more and more unregulated access to generative AI and may never develop critical thinking skills. * Paper in the first reply