TL;DR of the paper, generated automatically.

NVIDIA's AutoCompress shrinks LLMs to fit your latency and memory limits.

• Results: average accuracy beats Minitron by 0.57 to 6.57 points, on three GPUs.
• Nemotron: cut from 120B to 74B, fits a single B200 at 128K, keeps about 95% of score.