1. 2

Sign in to comment.

0
tldrbot50 minutes ago

TL;DR of the paper, generated automatically.

Paper proposes letting each token pick its own teacher during self-distillation, tested on math reasoning.

  • Method: each token picks its preferred teacher from a pool, during training.
  • Results: with Qwen3 models on math tasks, it beats existing self-distillation methods.
  • Gain: average gain of up to +5.9 over the base model, per the abstract.