通过位置嵌入锚点实现视频无缝循环生成
Loopy: Seamless Video Loop Generation via Anchored Looping Shift of Positional Embedding

- 发现DiT模型中不同层位置嵌入控制力不同,选定最强层作锚点
- 设计分层偏移策略,将时间感知从直线变为闭环,提升连贯性
- 支持身份控制与风格迁移,适用于游戏、社交等场景
循环视频在网页图形、游戏开发和社交媒体中至关重要。然而,现有方法因忽视视频生成模型对时间顺序的感知及其与循环行为的关系,常生成质量不佳的循环视频。本文首次揭示:DiT模型中不同注意力层的位置嵌入具有不同程度的时间控制能力,其中控制力最强的层可作为锚点。我们以该锚点为参考,为其余层提供强上下文先验,从而生成无缝且连贯的视频内容。基于此,提出分层位置嵌入偏移策略,根据各层时间控制效果设定特定偏移量,有效将DiT的时间感知从直线转换为环形。由此构建通用框架Loopy,支持RGB与RGBA视频生成,并具备身份控制与风格迁移等高级AIGC功能。实验表明,该方法显著提升生成循环视频的时间一致性和视觉保真度。模型已开源:https://donghaotian123.github.io/Loopy。
原文摘要 · Abstract (English)
Looping videos are essential for practical applications such as web graphics, game development, and social media. However, existing approaches typically fail to generate high-quality looping videos due to the neglect of how video generation models perceive temporal order and how this relates to the looping behavior. In this work, we are the first to reveal that position embedding at different attention layers within DiT exhibits varying levels of positional control, with the most pronounced layer acting as an anchor. We formulate this anchored layer as the reference point of the looping video, offering strong contextual priors for the remaining layers to facilitate the generation of seamless and coherent video content. Based on this insight, we propose an anchored position embedding shifting strategy that applies layer-specific shift lengths according to each layer's temporal control effect, effectively transforming DiT's temporal perception from a straight line to a circle. Leveraging this strategy, we develop a general framework, Loopy, for high-quality looping video generation, supporting both RGB and RGBA videos, while also enabling advanced AIGC features such as identity control and style transfer. Experiments demonstrate that our approach significantly improves temporal consistency and visual fidelity in generated looping videos. The released model is available on our website: https://donghaotian123.github.io/Loopy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。