arXiv:2507.04705cs.CV2025-07被引 5

通过解耦时空特征,实现人物身份一致的文本生成视频。

Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representations

  • 将提示词拆分为空间与时间两部分,分阶段生成图像与视频。
  • 在2025年ACM多媒体挑战赛中获亚军,身份保真度显著提升。
  • 适合需要人物形象稳定的视频生成应用,如虚拟主播、影视创作。

身份保真文本到视频生成(IPT2V)旨在生成高保真且人物形象一致的视频,对下游应用至关重要。然而,现有端到端框架存在严重的时空权衡:优化关键元素的空间一致性(如人物身份)常损害指令符合性的时间流畅性,而强调动态真实感又可能破坏视觉结构的空间连贯性。为此,我们提出一种简单有效的时空解耦框架,将表征分解为空间特征(用于布局)和时间特征(用于运动动态)。具体而言,论文设计了语义提示优化机制与分阶段解耦生成范式:前者将提示词解耦为空间与时间成分;后者分别由空间提示引导文本到图像阶段生成一致的空间特征,时间提示指导图像到视频阶段保证运动连续性。实验表明,该方法在时空一致性方面表现优异,身份保真度、文本相关性与视频质量均显著提升。基于此稳健机制,算法在2025年ACM MultiMedia Challenge中荣获亚军。代码已公开于https://github.com/rain152/IPVG。

原文摘要 · Abstract (English)

Identity-preserving text-to-video (IPT2V) generation, which aims to create high-fidelity videos with consistent human identity, has become crucial for downstream applications. However, current end-to-end frameworks suffer a critical spatial-temporal trade-off: optimizing for spatially coherent layouts of key elements (e.g., character identity preservation) often compromises instruction-compliant temporal smoothness, while prioritizing dynamic realism risks disrupting the spatial coherence of visual structures. To tackle this issue, we propose a simple yet effective spatial-temporal decoupled framework that decomposes representations into spatial features for layouts and temporal features for motion dynamics. Specifically, our paper proposes a semantic prompt optimization mechanism and stage-wise decoupled generation paradigm. The former module decouples the prompt into spatial and temporal components. Aligned with the subsequent stage-wise decoupled approach, the spatial prompts guide the text-to-image (T2I) stage to generate coherent spatial features, while the temporal prompts direct the sequential image-to-video (I2V) stage to ensure motion consistency. Experimental results validate that our approach achieves excellent spatiotemporal consistency, demonstrating outstanding performance in identity preservation, text relevance, and video quality. By leveraging this simple yet robust mechanism, our algorithm secures the runner-up position in 2025 ACM MultiMedia Challenge. Our code is available at https://github.com/rain152/IPVG.

视频生成身份保真时空解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。