无需训练即可提升文本引导视频生成的语义准确性
AlignVid: Training-Free Attention Scaling for Semantic Fidelity in Text-Guided Image-to-Video Generation
- 通过注意力重校准减少视觉主导干扰
- 在物体增删改任务中显著提升提示遵循度
- 适合关注生成精度与稳定性研究者
文本引导的图像到视频生成已取得显著进展，但在执行需大幅修改参考图像的任务（如物体增删改）时仍表现不佳。分析发现，这源于「视觉主导」现象：参考图像导致注意力过度分散，阻碍新语义信息的引入。为此，我们提出 AlignVid，一种无需训练的干预方法，通过能量视角重校模型内部注意力分布。AlignVid 采用注意力缩放调制（ASM）降低注意力熵，聚焦关键语义标记；结合引导调度（GS）保持生成稳定性。为严格评估性能，我们构建了 OmitI2V 基准，全面评测物体修改、添加与删除任务中的提示遵循能力。大量实验表明，AlignVid 在几乎无额外计算开销下显著提升语义保真度。代码与 OmitI2V 基准已开源。
原文摘要 · Abstract (English)
Text-guided image-to-video generation has made substantial progress, yet it still struggles to execute text-specified edits that require substantial changes to a reference image (\textit{e.g., object addition, removal, or modification}). Empirically, our analysis reveals that this stems from \textbf{visual dominance}, where the reference image causes severe attention dispersion, inhibiting the model's ability to incorporate new semantic information. To address this, we propose \textbf{AlignVid}, a training-free intervention that re-calibrates the model's internal attention distribution. Drawing on an energy-based perspective of attention, AlignVid employs Attention Scaling Modulation (\textbf{ASM}) to reduce attention entropy and concentrate focus on semantic tokens, alongside Guidance Scheduling (\textbf{GS}) to maintain generation stability. To rigorously assess this capability, we present \textbf{OmitI2V}, a comprehensive benchmark for evaluating prompt adherence across object modification, addition, and deletion. Extensive experiments demonstrate that AlignVid effectively enhances semantic fidelity with negligible computational overhead. Code and the OmitI2V benchmark are available at https://github.com/LAW1223/AlignVid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。