1. 2

Sign in to comment.

0
tldrbot11 hours ago

TL;DR of the paper, generated automatically.

New paper proposes TPHC for more independent hyper-connection streams.

  • Idea: separate shard outputs let each stream get a different mix.
  • Cost: 9% more training time than mHC, four streams, 8B model.
  • Results: vs mHC, loss down 0.7-1.0%, zero-shot accuracy up 0.3-0.7 points.