arXiv:2605.11927cs.CV2026-05

用物理模型提升多角色故事书生成的连贯性与动态性

RealDiffusion: Physics-informed Attention for Multi-character Storybook Generation

论文配图:RealDiffusion: Physics-informed Attention for Multi-character Storybook Generation
图 1 · 摘自论文原文
  • 引入热扩散先验,平滑序列特征抑制身份漂移
  • 区域感知随机过程防止故事僵化,保持姿态与场景变化
  • 无需训练的物理注意力机制,适合需要稳定叙事的生成任务

尽管现代扩散模型在生成单张图像方面表现优异,但将其扩展到序列生成时面临核心挑战:如何平衡叙事动态性与多角色一致性。现有方法常在二者间失衡,导致角色身份丢失或故事停滞。为此,我们提出RealDiffusion,一种统一框架,旨在实现强一致性与高动态性的协同。热扩散作为耗散先验,沿序列平均邻近特征并去除主体区域的高频噪声,抑制属性漂移,稳定角色身份。区域感知的随机过程引入微小扰动,探索附近模式,防止退化,确保姿态变化与场景演进。我们设计了一种轻量级、无需训练的物理感知注意力机制,在推理阶段将可调控的物理先验注入自注意力层。通过将特征演化建模为可配置的物理系统,该方法在不压制有意图的提示驱动变化的前提下,正则化时空关系。大量实验表明,RealDiffusion在角色一致性上显著优于现有最优方法,同时保持叙事动态性。代码已开源:https://github.com/ShmilyQi-CN/RealDiffusion。

原文摘要 · Abstract (English)

While modern diffusion models excel at generating diverse single images, extending this to sequential generation reveals a fundamental challenge: balancing narrative dynamism with multi-character coherence. Existing methods often falter at this trade-off, leading to artifacts where characters lose their identity or the story stagnates. To resolve this critical tension, we introduce RealDiffusion, a unified framework designed to reconcile robust coherence with narrative dynamism. Heat diffusion serves as a dissipative prior that averages neighboring features along the sequence and removes high-frequency noise within the subject region. This suppresses attribute drift and stabilizes identity across frames. A region-aware stochastic process then introduces small perturbations that explore nearby modes and prevent collapse so the story maintains pose change and scene evolution. We thus introduce a lightweight, training-free Physics-informed Attention mechanism that injects controllable physical priors into the self-attention layers during inference. By modeling feature evolution as a configurable physical system, our method regularizes spatio-temporal relationships without suppressing intentional, prompt-driven changes. Extensive experiments demonstrate that RealDiffusion achieves substantial gains in character coherence while preserving narrative dynamism, outperforming state-of-the-art approaches. Code is available at https://github.com/ShmilyQi-CN/RealDiffusion.

故事生成扩散模型多角色一致物理先验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。