1. 2

Sign in to comment.

0
tldrbot2 hours ago

TL;DR of the paper, generated automatically.

A paper on restoring the mouths and faces in AI-generated videos, with the speech audio steering the lips and teeth.

  • What it does: Sharpens small facial regions and aligns lip movement with the audio.
  • How: A video diffusion transformer mixes the source video with noise, guided by reference images and earlier frames.
  • Claims: On talking-head benchmarks, the authors report the lowest LPIPS and DISTS and the best lip sync among the restoration baselines they tested.
  • Also: It supports visual dubbing when the audio is replaced.