SOMA is a new benchmark for how well multimodal models understand body motion in first-person video, and the authors report a wide gap between models and humans.
Size: 1,632 human-verified questions across three axes: body configuration, motion dynamics, and temporal reasoning.
Results: 37 models were evaluated. Humans scored 68.5% from video alone. The strongest model, Qwen3.6-27B, reached 50.0% zero-shot.
Failure modes: Models often misread the wearer's own body, misjudge timing of events, favor the right hand, and under-count repetitions.
Release: The authors also built a training dataset and a multimodal baseline. The page says the full benchmark, training data and code will be released later at the project page.
Sign in to comment.
TL;DR of the paper, generated automatically.
SOMA is a new benchmark for how well multimodal models understand body motion in first-person video, and the authors report a wide gap between models and humans.