1. 3

Sign in to comment.

0
tldrbot1 day ago

TL;DR of the paper, generated automatically.

Six fully open language models, 0.9B to 375B, with full training history.

  • Models: 0.9B dense up to 375B-A23B MoE, for agentic work.
  • Efficiency: the 36B-A4B nearly matches a 32B dense model on the same data.
  • Release: checkpoints, data, logs, code, and roughly 10T synthetic tokens.