Distillation

A teacher-student training regime where a smaller model learns from supervision produced by a larger or stronger teacher model.

Distillation is a teacher-student training regime. A stronger teacher model produces supervision, and a smaller student model learns to match it so teams can deploy a cheaper model that keeps much of the teacher's useful behavior.

At a glance

Released

January 2024

Authors

Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, et al.

Regime type

Distillation

Related modules

No related modules listed yet.

What It Is

Distillation is a teacher-student training regime. A larger or stronger teacher model produces supervision, and a smaller student model learns to match that supervision. The teacher signal can appear as soft targets that spread probability across many plausible tokens, generated traces the teacher writes for the student to imitate, or labeled examples where the teacher marks the outputs the student should copy.

Why It Exists

Teams distill when they need a model they can run cheaply at scale. The student usually costs less per request, responds with lower latency than the full teacher, and fits deployment constraints such as smaller GPUs, on-device serving, or tighter serving budgets. Distillation trades extra training time and teacher compute for a model that keeps much of the teacher's useful behavior after deployment.

How It Works

The teacher model produces supervision such as soft targets, generated traces, or labeled examples. Those training examples or traces become the data the student learns from, the student weights are updated to match the teacher signal, and the trained student is the deployment-oriented model teams serve instead of the full teacher.
Distillation training flow
Request and weight flow
Distillation turns teacher supervision into training examples or traces, updates the student on that signal, and yields a model sized for deployment.
textdeploymentpayoffapproxfractextstudentinferencecosttextteachersupervisioncost\\text{deployment payoff} \\approx \\frac{\\text{student inference cost}}{\\text{teacher supervision cost}}

Compared To Nearby Regimes

Distillation is broader than any one variant. Offline distillation trains the student on a fixed or precomputed teacher data source: teacher outputs are collected once and replayed while the student learns. On-policy distillation refreshes supervision from behavior produced under the current student policy, so each update can use trajectories aligned with what the student would actually visit at inference time.

Limitations And Failure Modes

A student can only recover what the teacher signal actually covers. Weak supervision, narrow training data, or a student that is too small for the target behavior can all leave capability gaps.

Tags

References

  1. Agarwal, Rishabh, et al. "On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes." The Twelfth International Conference on Learning Representations, 2024, https://openreview.net/forum?id=3zKtaqxLhW.