Mismatch Matters: On-Policy Distillation Beyond Token Agreement
Quick summary
arXiv:2608.09836v1 Announce Type: new Abstract: On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repetitive loops to achieve near-perfect token agreement with the teacher despite globally flawed responses. We therefore shift our focus from agreement to teacher-student mismatch, and find that mismatch tokens can be mainly categorized into two types: student-excess tokens and student-deficit tokens. Student-excess tokens are generated by the student but assigned near-zero
Key takeaways
- arXiv:2608.09836v1 Announce Type: new Abstract: On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repetitive loops to achieve near-perfect token agreement with the teacher despite globally flawed responses.
- We therefore shift our focus from agreement to teacher-student mismatch, and find that mismatch tokens can be mainly categorized into two types: student-excess tokens and student-deficit tokens.
- Student-excess tokens are generated by the student but assigned near-zero
Why it matters
The significance is not only the legal text but how it changes product design. Decisions around “Mismatch Matters: On-Policy Distillation Beyond Token Agreement” may reshape data collection, model training, output accountability and market access.
Member comments