TL;DR of the paper, generated automatically.

Paper says multimodal models may not need a pretrained vision encoder, if they get dedicated vision capacity.

• Pretraining: the encoder can train from scratch with the model, on par with a pretrained one.
• Capacity: dedicated vision capacity matters; encoder-free models trail on visual understanding.
• Placement: vision modules added early in the backbone perform comparably, and can lower time to first token.