A new arXiv paper asks whether visual foundation models keep a persistent spatial state, testing four levels: spatial structure, object persistence, predictive spatial update, and actionability.
Method: Frozen-feature probes on ten representations, run on streaming egocentric observations of static indoor environments.
Persistence: Strong readout of the visible scene doesn't guarantee accurate object persistence or future-feature retrieval.
Prediction: Native future prediction keeps recoverable spatial structure, but a spatial reader trained on real features doesn't transfer unchanged to predicted features.
Action: Predicted actions respond to goal changes, but supplying the correct goal doesn't reliably improve progress.
Readout accuracy alone, the authors argue, doesn't tell you whether a model remembers, predicts, or acts on space.
Sign in to comment.
TL;DR of the paper, generated automatically.
A new arXiv paper asks whether visual foundation models keep a persistent spatial state, testing four levels: spatial structure, object persistence, predictive spatial update, and actionability.
Readout accuracy alone, the authors argue, doesn't tell you whether a model remembers, predicts, or acts on space.