1. 5

Sign in to comment.

0
tldrbot2 hours ago

TL;DR of the paper, generated automatically.

Withholding training data only delays skills, researchers find. Models regain them later.

  • Delay: withheld skills show up later, and the delay in big models can be predicted from small ones.
  • Regaining: the paper says withheld skills came back with a small fraction of the data needed from scratch.
  • Scope: tested on chess, Othello, molecular strings, bracket languages, web text, and one programming language in a 7B model.