arXiv:2608.20749cs.CV2026-08

用智能提示增强与语义修复,让视频生成更忠于人物身份和指令。

Identity-Preserving Text-to-Video Generation via Agentic Enhancement and Semantic Repair

论文配图:Identity-Preserving Text-to-Video Generation via Agentic Enhancement and Semantic Repair
图 1 · 摘自论文原文
  • 通过智能循环学习提示格式和人类生成经验,优化输入指令。
  • 识别生成视频中的错误区域,用视觉参考修复身份与语义偏差。
  • 轻量级多专家选择策略,提升输出可靠性,适合实际应用。

身份保持视频生成旨在根据自然语言指令生成视频,同时保留指定主体的视觉特征。现有商用模型虽具高画质与动作真实感,但仍存在身份漂移、指令遵循不全及复杂提示下细节缺失问题。由于这些模型通常为闭源黑箱,直接优化参数不可行。为此,我们提出轻量级增强框架AESR,包含全局智能提示增强模块:从官方文档学习模型特定提示格式,利用人机交互数据获取人类中心生成先验,并通过智能循环积累领域内身份保持生成经验形成可复用手册。此外,引入样本级视觉语义修复模块:使用视觉语言模型定位错误片段,设计修复指令,将选定帧转为显式视觉参考,并引导视频编辑模型修正局部语义或身份错误。还采用轻量级专家混合选择策略,从不同生成与修复路径中挑选可靠输出。在ACM MM 2026身份保持视频生成挑战赛官方评测协议下,我们的系统MIPL_Video在Track 1中排名第一,验证了AESR在实际场景中的有效性。代码已开源:https://github.com/oceanflowlab/AESR。

原文摘要 · Abstract (English)

Identity-preserving video generation aims to synthesize videos that follow natural-language instructions while maintaining the visual identity of a given subject. Recent commercial video generation models have achieved strong visual quality and motion realism, but they still suffer from identity drift, incomplete instruction following, and missing visual details under complex prompts. Since these models are usually closed-source black boxes, directly improving them through parameter optimization is often infeasible. We therefore propose Agentic Enhancement and Semantic Repair (AESR), a lightweight enhancement framework for identity-preserving video generation. To improve prompt construction before generation and mitigate the above failures, AESR introduces a global agentic prompt enhancement module. This module learns model-specific prompting formats from official documentation, acquires human-centered video generation priors from human-interaction data, and accumulates test-domain identity-preserving generation experience into a reusable playbook through an agentic loop. To further repair errors in videos generated with enhanced prompts, AESR introduces a sample-level visual semantic repair module, which uses a VLM to locate erroneous video segments and design repair instructions, edits selected frames into explicit visual references, and guides a video editing model to fix local semantic or identity-related errors. We also adopt a lightweight Mixture-of-Experts selection strategy to choose reliable outputs from different generation and refinement paths. Under the official evaluation protocol of the ACM MM 2026 Identity-Preserving Video Generation Challenge, our system MIPL\_Video ranked first in Track 1, demonstrating the effectiveness of AESR for practical identity-preserving video generation. The code is available at https://github.com/oceanflowlab/AESR.

视频生成身份保持智能增强语义修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。