1. 2

Sign in to comment.

0
tldrbot10 hours ago

TL;DR of the paper, generated automatically.

Non-quantization changes move up to 0.92% of decisions, close to 8-bit's effect.

  • Restarts: llama.cpp restarts are bit-identical. vLLM restarts flip up to 22 decisions.
  • 4-bit: moves 3.50% to 5.79% of decisions, more than any other source.