The paper argues that the stages of foundation-model training can all run under one likelihood-based algorithm.
The idea: RL objectives are derived as maximum-likelihood fitting to model-generated target distributions, which puts pretraining, midtraining, SFT and RL under one "unified training" method. The lifecycle becomes a question of which data to train on, in what proportions, and when.
The system: A single trainer mixes static data with RL generations.
Claimed results: The authors report gains in the accuracy-compute tradeoff and task coverage across controlled and real-world experiments, with broader capabilities preserved and a stronger start for continual learning.
Sign in to comment.
TL;DR of the paper, generated automatically.
The paper argues that the stages of foundation-model training can all run under one likelihood-based algorithm.