The comparison was with active learning
The researchers did not compare an AI tutor with a passive lecture. They compared it with classroom lessons that already used active-learning practices. Both formats covered the same undergraduate physics material. [1]
The 194 students took part in a randomized crossover design. Different groups used the tutor or attended class for different lessons, so each student could contribute evidence under both conditions. Pretests and post-tests measured learning for each lesson. [1]
More than an open chatbot
The tutor was built with carefully prepared prompts, explanations, questions and guardrails based on teaching research. It encouraged students to work through ideas and offered feedback at their own pace. The study therefore tests this particular teaching design, not any conversation with a general chatbot. [1]
Students showed larger short-term learning gains with the tutor while spending less time on the lesson. They also reported higher engagement and motivation. These are results from the study sessions, not proof that the tutor improves a full semester grade or long-term understanding. [1]
One successful lesson design does not settle education
The study took place in one Harvard physics course. The students, subject, assessment and tutor design limit how far the result can be generalized. Students knew they were in a study, and the comparison did not measure every benefit of learning with teachers and classmates. [1]
The practical lesson is narrower than replacing a classroom. An AI tutor can help when educators shape the material, interaction and feedback around a clear learning goal. Accuracy, access, teacher oversight and longer-term retention still need testing. [1]
Sources & context
One randomized crossover study in one undergraduate physics course.
AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting
Kestin and colleagues · Scientific Reports · June 3, 2025