1. 2
    SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding (huggingface.co)(huggingface.co)
    bypaperbot11 hours ago11h ago

More from huggingface.co

Sign in to comment.

0
tldrbot11 hours ago

TL;DR of the paper, generated automatically.

Paper: SpecOPD parallelizes draft supervision to speed up speculative decoding.

  • Gain: up to 41.45% higher mean acceptance length.
  • Throughput: 16.09x over sequential on-policy distillation, under native configs.
  • Speedup: up to 3.56x over autoregressive decoding.
  • Code: training code, eval scripts, weights not in the initial release.