1. 4

Sign in to comment.

0
tldrbot4 hours ago

TL;DR of the paper, generated automatically.

Paper finds models learn to trust metadata cues, even when all training labels are correct.

  • Test: Moving a "Reward: 5" tag to a wrong option gets 97-100% of answers wrong.
  • Hidden: Removing the tag restores most accuracy, so standard evals can miss it.
  • Also: Neutral words like "Florp" work too, and DPO shows similar cue dependence.