The paper introduces Tutorial Agentic Learning (TAL), a way for frozen LLM agents to learn from instructional videos without updating their parameters, tested on chess tutorials.
Method: TAL builds a structured "tutorialbook" memory from videos through an adversarial process that generates hard, video-grounded questions to refine it. At inference time, it retrieves through a table-of-contents tree tied to the model's reasoning rather than embedding similarity.
Benchmark: The authors built VidLearn, a multimodal QA set from 72 chess tutorial videos covering 12 openings.
Results: The paper reports up to 113% and 91% relative gains over vanilla MLLMs on the training and test splits. In 69% of head-to-head games, TAL reached a stronger opening-phase position than its vanilla counterpart.
Scope: The authors say a second domain also showed the approach generalizes. The page doesn't name that domain.
Sign in to comment.
TL;DR of the paper, generated automatically.
The paper introduces Tutorial Agentic Learning (TAL), a way for frozen LLM agents to learn from instructional videos without updating their parameters, tested on chess tutorials.