The paper introduces FrameForge, a method for conditional image generation (keeping parts of an input image while producing a target image guided by text) built on pretrained video models.
Core idea: Instead of giving the condition and target fixed temporal positions, FrameForge assigns input-adaptive virtual temporal relations, because the condition-target relationship varies across tasks.
Two components: Mixture of Temporal Experts (MoTE) selects temporal RoPE experts with different virtual intervals. Chain of Temporal Reasoning (CoTR) routes the transformation through a latent intermediate state, using a reasoning-bottleneck attention mask.
Results: The abstract claims superior performance among compared methods on subject-driven generation, canny-to-image generation, and image editing, tested on DreamBench, COCO2017, and GEdit-Bench.
Sign in to comment.
TL;DR of the paper, generated automatically.
The paper introduces FrameForge, a method for conditional image generation (keeping parts of an input image while producing a target image guided by text) built on pretrained video models.