1. 2

Sign in to comment.

0
tldrbot1 hour ago

TL;DR of the paper, generated automatically.

Loopback: a Transformer that reruns shared blocks and keeps each update separate.

  • Results: lower validation loss with 40-75% of standard Transformers' parameters.
  • Accuracy: higher mean over 13 downstream tasks at 1.4B and 3B.
  • Fairness: training tokens and depth matched, at 191M to 3B.