一个模型搞定透视与全景图的房间布局估计
uLayout: Unified Room Layout Estimation for Perspective and Panoramic Images
- 将两种图像统一映射到等距圆柱投影，实现无缝融合
- 通过1D卷积差异建模，适配不同视场角输入
- 首个端到端统一模型，实测在多个数据集表现优异
我们提出uLayout，一个统一模型，可从透视图和全景图中估计房间布局几何结构。传统方法需为每种图像类型设计独立模型，而uLayout将两类数据统一映射至等距圆柱投影，通过将透视图置于最优纬度坐标，实现跨域无缝利用。针对输入视场角（FoV）差异，设计共享特征提取器并加入额外1D卷积层，对不同域输入进行差异化条件处理，从而高效构建列式特征回归问题。该方法在不牺牲性能的前提下，首次实现单一端到端模型同时支持两种输入类型。在真实世界数据集LSUN、Matterport3D、PanoContext和Stanford 2D-3D上的大量实验验证了其有效性。代码已开源：https://github.com/JonathanLee112/uLayout。
原文摘要 · Abstract (English)
We present uLayout, a unified model for estimating room layout geometries from both perspective and panoramic images, whereas traditional solutions require different model designs for each image type. The key idea of our solution is to unify both domains into the equirectangular projection, particularly, allocating perspective images into the most suitable latitude coordinate to effectively exploit both domains seamlessly. To address the Field-of-View (FoV) difference between the input domains, we design uLayout with a shared feature extractor with an extra 1D-Convolution layer to condition each domain input differently. This conditioning allows us to efficiently formulate a column-wise feature regression problem regardless of the FoV input. This simple yet effective approach achieves competitive performance with current state-of-the-art solutions and shows for the first time a single end-to-end model for both domains. Extensive experiments in the real-world datasets, LSUN, Matterport3D, PanoContext, and Stanford 2D-3D evidence the contribution of our approach. Code is available at https://github.com/JonathanLee112/uLayout.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。