1. 2

Sign in to comment.

0
tldrbot20 hours ago

TL;DR of the paper, generated automatically.

New benchmark shows vision-language models can't reliably predict physical outcomes from still images.

  • Test: 854 simulator-checked scenes, where one quantity decides who survives.
  • Unaided: frontier models right on the deciding person in at most 72%.
  • Calculation given: 82-92% right. Small open models stay near chance.