改进视频外扩的掩码策略，提升模糊和不连贯问题
M3DDM+: An improved video outpainting by a modified masking strategy
- 训练时统一各帧掩码方向与宽度，解决训练推理不匹配
- 在相机运动少或外扩区域大时，视觉质量显著提升
- 保持原有效率，适合需要稳定视频生成的场景
M3DDM 通过潜在扩散建模实现了高效的视频外扩，但在相机运动受限或外扩区域较大等信息稀疏场景下，仍存在空间模糊和时间不一致的问题。我们发现其根源在于训练与推理阶段掩码策略不一致：训练时各帧采用随机掩码方向与宽度，而推理需保持一致的方向性外扩。为此，我们提出 M3DDM+，在训练中对所有帧应用统一的掩码方向与宽度，并对预训练的 M3DDM 模型进行微调。实验表明，M3DDM+ 在信息受限场景下显著提升了视觉保真度与时间连贯性，同时维持计算效率。代码已公开于 https://github.com/tamaki-lab/M3DDM-Plus。
原文摘要 · Abstract (English)
M3DDM provides a computationally efficient framework for video outpainting via latent diffusion modeling. However, it exhibits significant quality degradation -- manifested as spatial blur and temporal inconsistency -- under challenging scenarios characterized by limited camera motion or large outpainting regions, where inter-frame information is limited. We identify the cause as a training-inference mismatch in the masking strategy: M3DDM's training applies random mask directions and widths across frames, whereas inference requires consistent directional outpainting throughout the video. To address this, we propose M3DDM+, which applies uniform mask direction and width across all frames during training, followed by fine-tuning of the pretrained M3DDM model. Experiments demonstrate that M3DDM+ substantially improves visual fidelity and temporal coherence in information-limited scenarios while maintaining computational efficiency. The code is available at https://github.com/tamaki-lab/M3DDM-Plus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。