arXiv:2601.03655cs.CV2026-01被引 19

用记忆库实现跨镜头角色一致性生成

VideoMemory: Toward Consistent Video Generation via Memory Integration

论文配图:VideoMemory: Toward Consistent Video Generation via Memory Integration
图 1 · 摘自论文原文
  • 构建动态记忆库,存储角色/道具/场景的视觉语义特征
  • 在54个跨镜头场景中,实体一致性显著优于现有方法
  • 适合需要长视频连贯性的叙事生成任务

在叙事性视频生成中,保持角色、道具和环境在多镜头间的连续性是核心挑战。现有模型虽能生成高质量短片段,但在场景切换或实体长时间间隔再现时,常出现身份与外观不一致问题。我们提出VideoMemory,一种以实体为中心的框架,通过动态记忆库将叙事规划与视觉生成结合。给定结构化剧本,多智能体系统将叙事分解为镜头,从记忆库中检索实体表征,并基于这些表征生成关键帧与视频。动态记忆库存储角色、道具和背景的显式视觉与语义描述,并在每轮生成后更新,反映故事驱动的变化,同时保留身份。该检索-更新机制支持远距离镜头间实体的一致呈现,实现连贯的长序列生成。为评估此设定,我们构建了一个包含54个案例的多镜头一致性基准,涵盖角色、道具与背景持续场景。大量实验表明,VideoMemory在多种叙事序列中均实现了强实体级连贯性与高感知质量。

原文摘要 · Abstract (English)

Maintaining consistent characters, props, and environments across multiple shots is a central challenge in narrative video generation. Existing models can produce high-quality short clips but often fail to preserve entity identity and appearance when scenes change or when entities reappear after long temporal gaps. We present VideoMemory, an entity-centric framework that integrates narrative planning with visual generation through a Dynamic Memory Bank. Given a structured script, a multi-agent system decomposes the narrative into shots, retrieves entity representations from memory, and synthesizes keyframes and videos conditioned on these retrieved states. The Dynamic Memory Bank stores explicit visual and semantic descriptors for characters, props, and backgrounds, and is updated after each shot to reflect story-driven changes while preserving identity. This retrieval-update mechanism enables consistent portrayal of entities across distant shots and supports coherent long-form generation. To evaluate this setting, we construct a 54-case multi-shot consistency benchmark covering character-, prop-, and background-persistent scenarios. Extensive experiments show that VideoMemory achieves strong entity-level coherence and high perceptual quality across diverse narrative sequences.

视频生成一致性记忆机制叙事生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。