What researchers studied

Twelve successive randomized A/B tests comparing modifications with the active status-quo version of a targeted, phone-based numeracy tutoring program in Botswana. 14,818 students in grades 3 through 6 across 12 school terms in Botswana; trials ran from January 2021 through December 2024

What they found

  • Seven of the 12 tested modifications produced statistically significant efficiency gains, a 58 percent discovery rate within this program.
  • Pooled cost-reducing modifications lowered costs without a statistically detectable reduction in learning. A biweekly 40-minute model reduced costs by 11 percent, while next-available and centrally assigned tutor models reduced scheduling costs by 5 to 10 percent without reducing measured learning.
  • Randomly encouraging caregivers to co-lead part of the tutoring call produced intent-to-treat learning gains of 0.20 to 0.25 standard deviations relative to the active program model.
  • Several other inexpensive additions were less certain or ineffective: SMS practice and homework estimates were only marginally significant, while WhatsApp videos and motivational caregiver messages did not improve measured learning.

What the study does not prove

  • The experiments compared program modifications with an active status-quo tutoring model rather than with a no-tutoring control group, so they primarily identify relative efficiency and improvement rather than the standalone effect of tutoring.
  • The program served primary-school students in Botswana through short phone calls focused on foundational numeracy; results should not be generalized directly to U.S. private tutoring, other subjects, older students, or in-person instruction.
  • The 12 tests occurred sequentially across different school terms and evaluated different modifications, so the findings are not one single trial in which every option was compared simultaneously.
  • Learning was measured with short-term foundational numeracy assessments after roughly 12-week rounds; the study does not establish long-term retention, broader academic performance, or standardized-test effects.
  • Some low-cost add-on estimates used in the paper's efficiency discussion had p-values above conventional significance thresholds, and caregiver co-tutoring was a specific structured intervention rather than evidence that any form of parent involvement will improve learning.

Evidence strength: A large program of randomized operational experiments with directly measured numeracy outcomes and detailed cost analysis, strongest for comparing specific modifications within the Botswana phone-tutoring model.

Why this matters for families

Ask how a tutoring provider knows that a change actually helped. Strong programs should track delivered instruction and student learning, not assume that a new tool, schedule, or parent message works because it sounds promising.

Noor interpretation

How Noor translates the evidence into practice

Quality does not become visible through intention alone. A tutoring system learns when it tests operational assumptions, measures both learning and delivery, and keeps the changes that improve the student experience without weakening instruction.

Noor Lyra can use small, controlled quality-improvement cycles to evaluate scheduling, continuity, session completion, student effort, and family follow-through. Parent partnership should be structured around reinforcing the learning plan, while the educator remains responsible for diagnosis and instruction. Noor should not claim that these Botswana results predict its own effect sizes or that rotating tutors is always equivalent to continuity.

Read the original source

Noor links to the original or authoritative source so families can distinguish the evidence itself from our interpretation.

Open original source →

DOI: 10.26300/kaky-fr21

Research notes

tutoringprogram improvementparent partnershipimplementationrandomized experimentsnumeracy

No single study determines a student's plan. Noor uses research as one input alongside the learner's goals, observed performance, academic context, and response to instruction.