The paper argues that prediction error from video world models doesn't reliably measure physical surprise, because it also picks up appearance and motion.
Setup: The authors compare frozen representations from latent prediction, masked pixel reconstruction and simple visual statistics on IntPhys2, Physion++ and Kubric clips.
Support test: Across 256 configurations, V-JEPA2-G gives impossible motion lower error on solid support but higher error over a gap. A fixed cutoff reaches 53.9% balanced accuracy on clean clips but falsely flags about 69% of possible clips.
Other results: Repeating a single frame can raise error above the original clip, and in Kubric impossible events can produce lower error than possible ones.
Sign in to comment.
TL;DR of the paper, generated automatically.
The paper argues that prediction error from video world models doesn't reliably measure physical surprise, because it also picks up appearance and motion.