What researchers studied

Randomized field experiment assigning middle-school students to AI versus computer-assisted-learning-only support, mastery versus non-mastery progression, and one of two mathematics topics. More than 6,000 middle-school students in Hamilton County Schools

What they found

  • Students with AI support progressed more slowly and attempted fewer questions, but were more accurate conditional on reaching an attempt.
  • After mistakes, AI support improved next-attempt correctness and reduced the number of attempts needed to return to a correct answer while increasing time spent with structured support.
  • Mastery progression increased attainment of three correct answers in a row but did not by itself improve delayed learning.
  • The most encouraging delayed-test evidence was limited and concentrated on practiced material when AI was embedded in the mastery workflow.

What the study does not prove

  • This is an NBER working paper and should not be described as a peer-reviewed journal article.
  • The clearest evidence concerns the process of correction after mistakes, not a universal or large long-term achievement effect from AI tutoring.
  • The delayed-learning findings were more limited than the immediate next-attempt findings and depended on how AI support was structured.
  • The study does not establish that AI should replace educator judgment or that every AI tutor will produce the same effects.

Evidence strength: Strong experimental evidence; randomized field experiment; NBER working paper.

Why this matters for families

AI can support learning, but the study suggests that structure matters. The useful question is not whether a student has access to AI, but whether it helps the student reason through confusion without bypassing the work of learning.

Noor interpretation

How Noor translates the evidence into practice

The value of AI was not simply speed or answer access. Its strongest role appeared when structured support helped students work through an error and return to correct reasoning.

Noor Lyra should use AI as decision support inside a human-led learning process: observe the error, require learner reasoning, evaluate the recommendation, accept, modify, or reject it, and check later retention and transfer rather than immediate correctness alone.

Read the original source

Noor links to the original or authoritative source so families can distinguish the evidence itself from our interpretation.

Open original source →

DOI: 10.3386/w35621

Research notes

AI tutoringNBERmath learningmistakesmastery learninghuman oversight

No single study determines a student's plan. Noor uses research as one input alongside the learner's goals, observed performance, academic context, and response to instruction.