用Transformer重构屋顶线框，结合语义与几何信息提升精度。
Edge Prediction for Roof Wireframe Reconstruction with Transformers
- 基于DETR架构的端到端Transformer模型处理点云与语义图。
- 在HoHo 22k数据集上达到0.6476的混合结构分数，排名第二。
- 融合语义优先采样与自编码器特征，增强上下文理解。
本文针对S23DR Challenge 2026，提出一种从稀疏SfM点云及地面级语义分割图和深度图重建3D屋顶线框模型的方案。方法采用受DETR启发的端到端Transformer编码器-解码器结构。为有效处理几何与语义数据，稀疏SfM点云依据语义重要性动态下采样，并融合Gestalt与ADE20k类别特征；为进一步增强分割上下文，将点特征与通过冻结自编码器生成的潜在特征图投影得到的Gestalt特征编码进行融合。学习的查询嵌入通过交叉注意力机制直接解码为3D线框边。在HoHo 22k数据集上，该方法显著优于手工设计与学习型基线，取得0.6476的混合结构分数（HSS），位列挑战赛私有排行榜第二名。
原文摘要 · Abstract (English)
This paper presents a competitive solution to the S23DR Challenge 2026, which aims to reconstruct 3D house roof wireframe models from sparse SfM point clouds and ground-level semantic segmentations and depth maps. Our proposed method utilizes an end-to-end Transformer encoder-decoder architecture inspired by DETR. To effectively process the geometric and semantic data, the sparse SfM point cloud input is dynamically subsampled based on semantic priority and augmented with Gestalt and ADE20k class features. To further increase segmentation context, we fuse the point features with additional Gestalt feature encodings which are obtained by projecting the points into latent feature maps produced by a frozen autoencoder. Learned query embeddings are then decoded directly into 3D wireframe edges via cross-attention mechanisms. Evaluated on the "HoHo 22k" dataset, our approach significantly outperforms both handcrafted and learned baselines, achieving a Hybrid Structure Score (HSS) of 0.6476 and securing the second-highest position on the challenge's private leaderboard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。