The paper proposes Teacher-Evolution Self-Distillation (TESD), a fix for On-Policy Self-Distillation (OPSD) stalling during training.
The problem: OPSD often fails to keep improving, which the authors trace to the self-teacher giving unreliable guidance on some samples.
The fix: TESD optimizes the teacher using rewards for task correctness and context independence, then down-weights samples whose teacher guidance looks unreliable.
Results: The paper says TESD consistently beats representative token-level and sequence-level methods across multiple datasets and model scales. The author's comment names SciKnowEval and Tooluse, and says it beats SDPO and GRPO on them.
Sign in to comment.
TL;DR of the paper, generated automatically.
The paper proposes Teacher-Evolution Self-Distillation (TESD), a fix for On-Policy Self-Distillation (OPSD) stalling during training.