arXiv:2605.07800cs.CV2026-05

让视频生成更懂指令,精准对齐主体与背景关系。

SARA: Semantically Adaptive Relational Alignment for Video Diffusion Models

论文配图:SARA: Semantically Adaptive Relational Alignment for Video Diffusion Models
图 1 · 摘自论文原文
  • 根据文本语义动态分配注意力,优化关键对象间关系对齐
  • 在13维评估体系下超越SFT、VideoREPA等基线模型
  • 适合需要高精度语义控制的视频生成研究与应用

近期视频扩散模型虽能生成视觉逼真的视频片段,但仍存在遗漏实体、属性错配及交互关系弱化的问题。现有表示对齐方法如VideoREPA和MoAlign通过蒸馏冻结视觉基础模型中的时空标记关系来提升细粒度文本跟随能力,但其成对监督预算依据视觉或运动线索分配,而非与提示的相关性。本文提出SARA(语义自适应关系对齐),在保持冻结视频基础模型目标的标记关系蒸馏基础上,引入文本条件下的显著性机制,决定哪些标记对应接受监督。训练一个轻量级第一阶段对齐器,使用每实体的SAM 3.1掩码监督和InfoNCE正则项,并通过一对路由算子将连续显著性融合进蒸馏过程:当任一端点显著时,该标记对获得权重,从而将监督导向主体-主体与主体-背景对,避开背景-背景对。在Wan2.2持续训练设置下,SARA在13维视觉语言模型评估体系、公开的VBench基准以及盲测用户研究中均优于SFT、VideoREPA和MoAlign。

原文摘要 · Abstract (English)

Recent video diffusion models (VDMs) synthesize visually convincing clips, yet still drop entities, mis-bind attributes, and weaken the interactions specified in the prompt. Representation-alignment objectives such as VideoREPA and MoAlign improve fine-grained text following by distilling spatio-temporal token relations from a frozen visual foundation model, but their pairwise supervision budget is allocated by visual or motion cues rather than by how relevant each pair is to the prompt. We present SARA, Semantically Adaptive Relational Alignment, which keeps token-relation distillation (TRD) on a frozen VFM target and adds a text-conditioned saliency that decides which token pairs carry supervision. A lightweight Stage~1 aligner is trained with per-entity SAM~3.1 mask supervision and an InfoNCE regulariser, and its continuous saliency is fused into TRD through a pair-routing operator that assigns each token pair a weight whenever either of its two endpoints is salient, thereby routing supervision toward subject-subject and subject-background pairs and away from background-background ones. In the Wan2.2 continual-training setting, SARA improves both text alignment and motion quality over SFT, VideoREPA, and MoAlign on a 13-dimension VLM rubric, on the public VBench benchmarks, and in a blind user study. Project page: https://saradit.github.io/.

视频生成扩散模型文本对齐关系建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。