A new paper introduces YANAMI, a dataset and benchmark for multimodal dialogues where the model must track and update the conversation state across turns.
Dataset: YANAMI-400K has 400K multi-turn multimodal dialogues, with over 2M images and 2.6M turns, across four scenarios: Multi-Image Fusion, Image Editing, World Modeling and Visual Reasoning.
Benchmark: YANAMI-Bench has 400 multi-turn dialogues and jointly evaluates Dialogue State Understanding and Visual Realization.
Results: Per the paper, fine-tuning BAGEL on YANAMI-400K improves image editing by 10.3%, generation by 2.0% and stateful interaction by 13.23%, while keeping general visual understanding. The authors say existing unified multimodal models still struggle on YANAMI-Bench.
Sign in to comment.
TL;DR of the paper, generated automatically.
A new paper introduces YANAMI, a dataset and benchmark for multimodal dialogues where the model must track and update the conversation state across turns.