TL;DR of the paper, generated automatically.

LoopVQ shrinks the KV cache of looped transformers to 0.25 to 0.5 bits.

• Method: codes channels from several loop steps jointly, codebooks fine-tuned.
• Accuracy: beats prior methods at 0.5 bits, keeps much BF16 accuracy at 0.25.
• Speed: nearly halves decode latency at 32K; up to 6.1x throughput on Ouro-2.6B.