arXiv:2605.21131cs.CV2026-05

统一视觉几何感知,支持在线与离线模式无缝切换。

UniT: Unified Geometry Learning with Group Autoregressive Transformer

论文配图:UniT: Unified Geometry Learning with Group Autoregressive Transformer
图 1 · 摘自论文原文
  • 用分组自回归机制统一处理多视角传感器数据
  • 在10个基准上实现当前最优的联合几何感知性能
  • 适合需要多场景、长时序几何理解的研究者

近期的前馈模型显著推进了从传感器观测中推断密集3D结构的几何感知能力。然而,其核心能力仍分散在多个不兼容范式中,包括在线感知、离线重建、多模态融合、长时序可扩展性及度量尺度估计。我们提出UniT,一种基于新型分组自回归变压器的统一模型,将这些看似不同的能力整合到单一框架中。核心思想是将传感器观测组作为基本自回归单元,以无锚点、自适应尺度的方式预测对应点图。无论是在线还是离线设置下的多种视图配置,均可自然地统一在一个分组自回归过程中。通过调整组大小,在线模式采用单帧组进行多步自回归,而离线模式则在一次前向传播中聚合多帧组。同时,基于队列的键值缓存机制确保了长时序下自回归内存的有界性,通过无锚点关系建模降低对早期帧的长程依赖,使过时记忆可实时丢弃。为提升跨场景的度量尺度泛化能力,引入自适应尺度几何损失,将相对几何约束与部分绝对尺度项结合,隐式正则化全局尺度并诱导从尺度不变几何到度量尺度解的渐进过渡。配合专用模态注意力模块实现辅助模态融合,UniT在涵盖七个代表性任务的十个基准上实现了当前最优性能。

原文摘要 · Abstract (English)

Recent feed-forward models have significantly advanced geometry perception for inferring dense 3D structure from sensor observations. However, its essential capabilities remain fragmented across multiple incompatible paradigms, including online perception, offline reconstruction, multi-modal integration, long-horizon scalability, and metric-scale estimation. We present UniT, a unified model built upon a novel Group Autoregressive Transformer, which reformulates these seemingly disparate capabilities within a single framework. The key idea is to treat groups of sensor observations as the basic autoregressive units and predict the corresponding point maps in an anchor-free and scale-adaptive manner. More specifically, diverse view configurations in both online and offline settings are naturally unified within a single group autoregression process. By varying the group size, online mode operates over multiple autoregressive steps with single-frame groups, whereas offline mode aggregates a multi-frame group in a single forward pass. Meanwhile, a queue-style KV caching mechanism ensures bounded autoregressive memory over long horizons. This is enabled by reducing long-range dependencies on early frames through anchor-free relational modeling, thereby allowing outdated memory to be discarded on the fly. To improve metric-scale generalization across scenes, a scale-adaptive geometry loss is further introduced within this framework. It couples relative geometric constraints with a partial absolute scale term, implicitly regularizing global scale and inducing a progressive transition from scale-invariant geometry to metric-scale solutions. Together with a dedicated modal attention module for integrating auxiliary modalities, UniT achieves state-of-the-art performance in unified geometry perception, as validated on ten benchmarks spanning seven representative tasks.

几何感知自回归多模态3D重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。