This is a survey and position paper arguing that embedding research across modalities is converging on shared principles, and proposing a framework to unify it.
The framework: Three capabilities, coverage, comparability and compositionality, each harder than the last, reached through three paths: transformation, alignment and joint embedding.
The advocated end point: Joint embedding, which the paper says aims at compositionality and subsumes alignment.
Shared ground: Contrastive learning, Transformer architectures and a shift toward whole semantic units, across text, vision, audio, graphs, tables and time series.
What it still needs: Preserving modality-specific structures, cross-pollination between research communities, and unified benchmarks that measure compositional gains.
Sign in to comment.
TL;DR of the paper, generated automatically.
This is a survey and position paper arguing that embedding research across modalities is converging on shared principles, and proposing a framework to unify it.
Three paths, three capabilities, one terminus.