用旋转等变性简化全景世界模型的长程记忆问题。
PanoWorld: Real-World Panoramic Generation
- 通过固定朝向将相机轨迹转为平移，利用全景射线条件与几何感知记忆增强。
- 在World360数据集上显著优于现有方法，尤其在大尺度空间变化下表现突出。
- 适合做全景生成、自动驾驶场景建模的研究者参考。
本文针对全景世界模型中的长程记忆挑战，利用全向表示的旋转等变特性，将旋转视为隐式几何变换。基于此，提出PanoWorld，通过固定朝向将相机轨迹简化为平移，结合密集全景射线条件（DPRC）和几何感知记忆增强（GMA），实现当前动作建模与长程记忆的统一。设计了三阶段训练流程，逐步优化各模块。为更好评估在大规模空间变化与多样光照条件下的物理一致性，构建了包含真实无人机采集视频与AirSim360生成高质量模拟片段的大规模数据集World360。在World360上的大量实验表明，PanoWorld显著优于现有方法。相关模型、代码与数据集将公开。更多信息见项目页：https://lihaoy-ux.github.io/panoworld-page/
原文摘要 · Abstract (English)
In this work, we aim to address the challenge of long-range memory in panoramic world models by exploiting the rotation-equivariant property of omnidirectional representations, where rotation can be treated as an implicit geometric transformation.Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware Memory Augmentation (GMA).Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse illumination conditions, where existing datasets are relatively stable, we construct World360, a large-scale dataset consisting of both real-world video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by AirSim360.Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large margin.Our models, training code, and dataset will be publicly available. More information can be found on our project page: https://lihaoy-ux.github.io/panoworld-page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。