让大模型在4K全景图上实现精准实例分割，性能比普通模型高17.2%。
SAP: Segment Any 4K Panorama
- 将全景图拆成连续视角片段，用视频方式处理提升稳定性
- 合成18.3万张带标签的4K全景图训练模型，支持高分辨率输入
- 零样本迁移效果显著，适合增强现实与机器人视觉场景
提示驱动的实例分割广泛应用于具身智能与AR系统中，但基于透视图像训练的基础模型在360°全景图上性能常下降。本文提出面向4K高分辨率全景图的实例分割基础模型SAP。我们将全景分割重构为固定轨迹的透视视频分割任务，将全景图分解为沿球面连续遍历采样的重叠透视块。该记忆对齐的重构方式在保持原生4K分辨率的同时，恢复了稳定跨视角传播所需的平滑视点过渡。为实现大规模监督，我们利用InfiniGen引擎合成183,440张4K分辨率全景图像并附带实例分割标签。在该轨迹对齐范式下训练的SAP，在真实世界4K全景基准上相较不同规模的vanilla SAM2实现+17.2%的零样本mIoU提升。
原文摘要 · Abstract (English)
Promptable instance segmentation is widely adopted in embodied and AR systems, yet the performance of foundation models trained on perspective imagery often degrades on 360° panoramas. In this paper, we introduce Segment Any 4K Panorama (SAP), a foundation model for 4K high-resolution panoramic instance-level segmentation. We reformulate panoramic segmentation as fixed-trajectory perspective video segmentation, decomposing a panorama into overlapping perspective patches sampled along a continuous spherical traversal. This memory-aligned reformulation preserves native 4K resolution while restoring the smooth viewpoint transitions required for stable cross-view propagation. To enable large-scale supervision, we synthesize 183,440 4K-resolution panoramic images with instance segmentation labels using the InfiniGen engine. Trained under this trajectory-aligned paradigm, SAP generalizes effectively to real-world 360° images, achieving +17.2 zero-shot mIoU gain over vanilla SAM2 of different sizes on real-world 4K panorama benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。