If you want to get a glimpse of the cutting-edge research on artificial intelligence in education, look to Jing Liu, director of the Center for Educational Data Science and Innovation at the University of Maryland. Two studies published in October offer a striking and illuminating contrast. It has been suggested that a little AI-powered math practice could help middle school students in difficulty improve their skills. The other found that an AI assistant was detrimental to student engagement and harm their grades.
Liu wanted the studies to be published together, he told me, to show that AI in education is neither “all good” nor “all bad.” The field is still seeking to design tools that improve learning over time. And that means studying not just the technology, but how students respond to it — including in ways its creators never anticipated.
In the first experienceA math practice website called Goblins was rolled out to 6,000 middle school students in Deer Valley, Arizona, during the 2025-2026 school year. Teachers assigned problems to their students. An AI tutor offered step-by-step help via text chat, drawing on a virtual whiteboard, and even a spoken conversation. Teachers were encouraged to use it with struggling students, although some assigned it to their entire classes and others ignored it altogether.
Students who solved at least two problems per week – more than 50 during the school year – performed significantly better on a standardized end-of-year math assessment. than similar students who used goblins less or not at all. Accelerate, an independent, The nonprofit research organization that commissioned the study equated the statistical difference of 0.11 standard deviations to approximately three additional months of training.
But wait for the robot party music. The study cannot prove that goblins are behind the gains. Students who used it more may simply have been more motivated to practice math and may well have performed better in math with or without goblins. Accelerate hopes to commission a randomized controlled trial to settle the matter.
However, one result of this study is worth highlighting: half of the 6,000 students used Gobelins to solve at least two problems per week, a dose associated with higher math scores.
This is notable because educational technology is plagued by what education researchers call “five percent problem.” A tool may work wonderfully for students who use it as directed, but typically only about 5% actually do so. The other 95 percent either don’t log in at home or sit idly during scheduled minutes of screen time in class. Studies on Khan Academy, IXL and other platforms have well documented this challenge of getting kids to use their educational software consistently.
Goblins seem to reach some students who are often difficult to engage, including English learners and students with disabilities. Liu suspects it’s because students could speak out loud to a cartoon avatar of their teacher, who could respond. Additionally, the AI tutor could speak and understand Spanish and six other languages.
“It was like talking to your own teacher,” Liu said. “It made students want to use it.”
Teacher control over homework may also have been helpful. Rather than handing over the lesson to a computer, teachers decided which problems to assign to which students and had the option to upload their own problems. This echoes research findings on older educational technologies. Another college math platform, ASSISTments, also produced solid results using relatively small amounts of teacher-assigned practice with instant feedback for students.
Still, I don’t want to overstate how attractive or appealing the goblins were. Only 6 percent of students in this experiment used it at least half an hour per week, which is a more common dosage recommendation in educational technology and for which use of Goblins reflects the familiar five percent problem.
When access to AI backfires
Liu’s other experiment tested an AI assistant developed at the University of Maryland. Before rolling it out university-wide to all 40,000 students, Dean Jennifer King Rice asked Liu to study it first.
During the fall 2025 semester, Liu 15 teachers out of 30 randomly assigned to add the AI assistant to their classes. The other 15 taught the same courses, without AI. Subjects ranged from English and mathematics to biology and social sciences. For example, two professors taught separate sections of the same biology course, one with the AI assistant and one without.
Teachers provided the university’s AI chatbot with syllabi, readings, assignments, and other course materials. During the semester, students can ask questions about assignment instructions and deadlines or get help understanding concepts. The assistant was available 24 hours a day through Canvas, the university’s online learning management system.
At the end of the year, the 1,100 students taking courses with the AI assistant had lower grades—reduced by an average of 0.11 points on a four-point scale, or roughly the difference between a 3.0 (a strong B) and a 2.9 (a weak B)—than the 1,200 students in the control group. They also interacted less with course material (e.g., by making a comment on the class’s online discussion board) and were less likely to attend office hours or meet with instructors and peers. The negative effects were more pronounced for first-generation students.
“AI tools could exacerbate inequalities in learning outcomes,” Rice said at a news conference. University of Maryland Publication. “This particularly bothers me.”
Only 15% of students in classes using AI have ever used the assistant. Those who did mostly asked it for answers rather than taking advantage of its tutoring features.
How can a tool used by so few students have such negative effects?
Liu and Rice suspect that the AI assistant’s constant presence on the course website sent an unintended message: AI use is encouraged. Students may have taken this as permission to use ChatGPT, Claude, or Gemini outside of the university’s platform to complete their assignments. Researchers cannot establish whether this occurred. But if AI did the work for students, it might have allowed them to learn less.
The lesson is not that universities should avoid AI, Liu says. The fact is that even a carefully designed tool can have consequences beyond its intended use. And Maryland deserves credit for testing the assistant before its university-wide rollout and making the troubling results public so the entire field can learn from it.
Contact the editor Jill Barshay at 212-678-3595, jillbarshay.35 on Signal, or barshay@hechingerreport.org.
This story on the search for AI tutors was produced by The Hechinger reportan independent, nonprofit news organization that covers education. Register for Proof points and others Hechinger Newsletters.
The article Same researcher, opposite results appears first on The Hechinger Report.
