Generative AI in education

An LLM tutor helps when code checks answers, a student model such as knowledge tracing decides when to reveal one, and hints come first. In a 2024 field study, free answer access raised practice scores but cut exam scores.

8 min read825 wordscore0 viewsupdated 2026-10-21

Definition

Generative AI in education is the use of generative models to tutor students, give feedback on their work, generate practice material and help teachers plan. The design target is Bloom's "2 sigma problem" (Bloom, Educational Researcher 1984): students tutored one-to-one scored about two standard deviations above students taught in a conventional class, and no school can afford a tutor for every student. The main risk is the opposite effect: a model that does the work for students, so they practise less and learn less.

A chat model answers questions by default. A tutor should mostly not answer them. It should find the student's mistake, give the smallest hint that lets the student fix it, and track which skills the student has mastered. A general assistant does none of this, so an education product wraps the model in a system prompt, tools and a student model (see System prompts and role prompting).

Two facts shape the design. First, LLMs make arithmetic and reasoning slips, so the correctness of a student's answer should come from a solver, a unit test or an answer key called as a tool (see LLM APIs and function calling), not from the model's judgement. Second, the decision to reveal an answer is a teaching decision. It belongs in explicit policy driven by a student model, not in the sampling of the next token.

The tutor loop

AI tutor turn: check the work, update the student model, give the smallest useful hintstudenttask: solve 3x + 5 = 20answer: x = 12(15 − 3 instead of 15 ÷ 3)1 answer check3·12 + 5 = 41 ≠ 20by a solver tool,not by the LLM2 student model (BKT)skill: 2-step equationsP(L): 0.20 → 0.18 (wrong)4 tutor policy(system prompt + rules)never lead with answerone hint rung per turnuse course materialsgrade-level languageflag distress to teacher5 teacher dashboardper-skill P(L), hint rungs used,answers given away, flagged chats6 student dataFERPA / COPPA / GDPR: collect theminimum; contract bars training use3 hint ladder (hover a rung; tutor starts at L1)L1 nudgeWhat do you get if you put x = 12 back in?L2 strategyUndo the + 5 first, then the × 3.L3 worked step3x + 5 − 5 = 20 − 5, so 3x = 15.L4 answerx = 5, logged as AI-given (A)hint back to studentthe LLM writes the hint; correctness, mastery and when to reveal the answer come from code
fig.One tutor turn. The student answers x = 12 to 3x + 5 = 20 (15 − 3 instead of 15 ÷ 3). A solver tool finds 3·12 + 5 = 41 ≠ 20. Knowledge tracing lowers P(mastered) from 0.20 to 0.18. The tutor policy picks the lowest rung of the hint ladder, a nudge to substitute the answer back. The full answer (L4) is given only after the lower rungs fail, and it is logged as AI-given. A teacher dashboard and data-protection rules sit under the whole loop. Hover a rung to highlight it.

What runs on each student turn

  1. 1Check. Grade the answer with a deterministic tool: a computer algebra system, the course's unit tests for code, or an answer key. The LLM gets the verdict and the error as input.
  2. 2Diagnose. The LLM, prompted with the task, the student's working and the verdict, names the likely misconception (here, inverting × 3 with − 3).
  3. 3Update the student model. Update the mastery estimate for the skill using knowledge tracing (below). Answers the tutor gave away do not count as evidence of mastery.
  4. 4Choose the hint rung. The policy moves up a ladder: nudge, strategy, worked step, answer. It starts at the lowest rung that has not been tried and climbs only after another failed attempt.
  5. 5Write the reply. The LLM writes the hint at the chosen rung, in grade-level language, grounded in the course materials (see RAG architecture). A filter checks that it does not contain the final answer before L4.
  6. 6Report. Mastery per skill, rungs used and flagged conversations go to the teacher's dashboard.

Knowing when the student has learned it

Bayesian Knowledge Tracing (BKT; Corbett and Anderson, 1995) is the classic student model of intelligent tutoring systems. Each skill is a hidden binary state, learned or not. Four parameters per skill: the prior P(L0)P(L_0), the chance of a lucky guess gg, the chance of a slip ss when the skill is known, and the chance TT of learning it at each practice opportunity.

P(L∣✓)=P(L)(1−s)P(L)(1−s)+(1−P(L))g,P(L∣×)=P(L) sP(L) s+(1−P(L))(1−g)P(L\mid\checkmark)=\frac{P(L)(1-s)}{P(L)(1-s)+\big(1-P(L)\big)g},\qquad P(L\mid\times)=\frac{P(L)\,s}{P(L)\,s+\big(1-P(L)\big)(1-g)}
P(L)P(L)
current probability that the student has mastered the skill
g, sg,\ s
guess and slip probabilities: correct without the skill, wrong with it
P(Ln+1)=P(L∣obsn)+(1−P(L∣obsn)) T,P(correctn+1)=P(Ln+1)(1−s)+(1−P(Ln+1))gP(L_{n+1})=P(L\mid\text{obs}_n)+\big(1-P(L\mid\text{obs}_n)\big)\,T,\qquad P(\text{correct}_{n+1})=P(L_{n+1})(1-s)+\big(1-P(L_{n+1})\big)g
TT
probability of learning the skill at one practice opportunity
0.950.95
the usual mastery threshold in Cognitive Tutor-style systems: move on once P(L) ≥ 0.95

The trap for an LLM tutor is the answer it gives away. If the model shows the solution and the student copies it, the log contains a correct answer that carries no evidence of skill. ASSISTments, a widely used maths tutoring platform, scores any problem where the student asked for the answer as incorrect. Score it as correct and the student model declares mastery too early, so the tutor stops teaching.

student model: P(skill mastered) after each attempt · AI-given answers scored as incorrect00.510481216attemptmastery 0.95✗AA✓✓✗✓✓✓✓after 10 attemptsP(L) now0.999P(next correct)0.899mastered at#8other scoring#4tutor's next moveP(L) ≥ 0.95: advance to thenext skill, fade hintsviolet: this scoringdashed: the other one✓ correct unaided · ✗ wrong · A correct only after the AI gave the answerP(L|✓) = P(L)(1−s) / [P(L)(1−s) + (1−P(L))g], then P(L) ← P(L|obs) + (1 − P(L|obs))·Tg = 0.20, s = 0.10, T = 0.15, P(L₀) = 0.20
add attempt
score A as
fig.Defaults: sequence ✗ A A ✓ ✓ ✗ ✓ ✓ ✓ ✓ with P(L₀) = 0.2, g = 0.2, s = 0.1, T = 0.15, and AI-given answers (A) scored as incorrect. P(L) stays at 0.17 through the two given answers, rises to 0.874 after two unaided correct answers, drops to 0.544 at the next mistake and crosses 0.95 at attempt 8 (0.972). Scored naively as correct, the same log reaches mastery at attempt 4 (0.974) after one unaided answer. Build ✗ A A A A A: naive scoring declares mastery at attempt 4 with no unaided answer at all, while strict scoring settles at 0.171. Raise g to 0.35 and strict mastery moves to attempt 9.
#ObservedA scored incorrect: P(L|obs) → P(L next)A scored correct: P(L|obs) → P(L next)
1✗ wrong0.030 → 0.1760.030 → 0.176
2A answer given0.026 → 0.1720.490 → 0.566
3A answer given0.025 → 0.1720.855 → 0.876
4✓ correct0.482 → 0.5600.970 → 0.974 (mastered)
5✓ correct0.851 → 0.8740.995 → 0.995
Worked example: the first five attempts of the default log (g = 0.2, s = 0.1, T = 0.15, P(L₀) = 0.2), computed in Python

Row 1 in full: P(L∣×)=0.2⋅0.10.2⋅0.1+0.8⋅0.8=0.020.66=0.030P(L\mid\times)=\frac{0.2\cdot0.1}{0.2\cdot0.1+0.8\cdot0.8}=\frac{0.02}{0.66}=0.030, then 0.030+0.970⋅0.15=0.1760.030+0.970\cdot0.15=0.176. Under strict scoring, attempt 8 is the first to pass 0.95, after three unaided correct answers in a row.

python
1def bkt_update(p, correct, guess=0.2, slip=0.1, learn=0.15):
2 """One Bayesian Knowledge Tracing step: Bayes on the answer, then learning."""
3 if correct:
4 post = p * (1 - slip) / (p * (1 - slip) + (1 - p) * guess)
5 else:
6 post = p * slip / (p * slip + (1 - p) * (1 - guess))
7 return post + (1 - post) * learn
8
9def mastery_trace(attempts, p0=0.2, assisted_counts_as=False):
10 """attempts: 'C' correct unaided, 'W' wrong, 'A' correct after the AI gave the answer."""
11 p, out = p0, []
12 for a in attempts:
13 correct = a == "C" or (a == "A" and assisted_counts_as)
14 p = bkt_update(p, correct)
15 out.append(round(p, 3))
16 return out
17
18log = "WAACCWCCCC"
19print(mastery_trace(log))
20# [0.176, 0.172, 0.172, 0.56, 0.874, 0.544, 0.867, 0.972, 0.995, 0.999]
21print(mastery_trace(log, assisted_counts_as=True))
22# [0.176, 0.566, 0.876, 0.974, 0.995, 0.967, 0.994, 0.999, 1.0, 1.0]
BKT over a tutoring log, scoring AI-given answers both ways (plain Python, runs as is; reproduces the table and the figure's defaults)

Evidence so far

Results to know, each in its own setting

Bloom (1984)
One-to-one tutoring with mastery learning: the average tutored student scored about 2 SD above the conventional class mean, above roughly 98% of that class (Φ(2) = 0.977).
Bastani et al. (2024)
Field experiment with about 1,000 Turkish high-school maths students. Unrestricted GPT-4 access raised practice-problem scores by 48% but lowered later exam scores (taken without AI) by 17%. A tutor version with hints and teacher-written answers raised practice scores by 127% and largely removed the harm.
AI-text detectors
OpenAI withdrew its own AI-text classifier in July 2023 for low accuracy. It had flagged 26% of AI-written text, with 9% false positives on human text. Liang et al. (2023) found popular detectors labelled more than half of TOEFL essays by non-native English writers as AI-generated.
Khanmigo (2023)
Khan Academy's GPT-4-based tutor, launched in 2023, is designed to guide students with questions rather than hand over answers. It is an early large deployment of the hint-first pattern.

Uses beyond tutoring

Where it's used

Feedback on writing and code

Rubric-anchored comments on a draft, with the rubric and exemplars in the prompt. Grades stay with the teacher; scores from an LLM judge drift with prompt wording (see LLM-as-a-judge).

Practice and item generation

Variants of worked problems, distractors for multiple-choice items, reading passages at a target level. Verify answer keys with a solver and have a teacher review the items before use (see Synthetic data generation).

Teacher planning

Lesson outlines, differentiated versions of a worksheet, parent letters in several languages: drafts the teacher edits.

Language and accessibility

Conversation practice, text simplification, read-aloud and translation for students who need them (see Conversational assistants).

Use when

  • +Patient, always-available practice with immediate feedback, at a cost far below a human tutor.
  • +Hints can target the specific misconception in the student's working.
  • +Teachers get per-skill mastery data instead of a single grade.

Avoid when

  • −Answer-giving replaces practice; students score higher with the tool and lower without it.
  • −Confident wrong explanations, especially in maths, unless a tool checks every answer.
  • −Bias in feedback and examples across student groups (see Bias and fairness in generative models).
  • −Children's data, consent and age rules add legal and contractual work.

Check yourself

An LLM tutor reveals the solution and the student types it in. How should a knowledge-tracing student model treat that attempt?

Check yourself

With g = 0.2, s = 0.1 and current P(L) = 0.5, what is the posterior P(L) after one correct answer (before the learning step)?

Remember

  • ✓A tutor is a policy around the LLM: tool-checked answers, a student model, a hint ladder (nudge, strategy, worked step, answer) and teacher reporting.
  • ✓Bayesian Knowledge Tracing: Bayes update with guess gg and slip ss, then learning TT; advance once P(L)≥0.95P(L)\ge0.95.
  • ✓Never count an answer the AI gave away as evidence of mastery; in the figure that moves mastery from attempt 8 to 4.
  • ✓Bloom's 2-sigma result is the goal; Bastani et al. (2024) showed unrestricted answer access can raise practice scores and lower exam scores.
  • ✓AI-text detectors are unreliable and biased against non-native writers; redesign assessment rather than police it.
  • ✓FERPA, COPPA, GDPR and the EU AI Act's high-risk list for grading and admissions govern deployment; confirm the current rules.
← back to Generative AI roadmap