arXiv:2605.03359cs.CV2026-05International Conf…被引 1

融合前馈重建与生成式3D先验,实现对齐的多视角3D重建与姿态估计。

Mix3R: Mixing Feed-forward Reconstruction and Generative 3D Priors for Joint Multi-view Aligned 3D Reconstruction and Pose Estimation

论文配图:Mix3R: Mixing Feed-forward Reconstruction and Generative 3D Priors for Joint Multi-view Aligned 3D Reconstruction and Pose Estimation
图 1 · 摘自论文原文
  • 分两阶段生成:稀疏体素+纹理几何,用混合注意力机制对齐2D-3D
  • 在DTU和BlendedMVS上,3D对齐度优于纯生成方法,姿态估计精度超前馈方法
  • 无需微调即可将输入纹理精准贴合生成形状,适合高精度3D重建场景

稀疏视图3D重建近年发展出两条路径:前馈重建能生成像素对齐的点图但缺乏完整几何结构,生成式方法虽可生成完整几何却常出现输入对齐不佳。本文提出Mix3R,将前馈重建与3D生成融合于统一框架中实现对齐。其分两阶段生成:第一阶段生成稀疏体素、每视点点图及相机参数,三者均对齐至同一3D结构;通过引入混合变换器架构,在前馈模型与预训练3D生成模型中插入全局自注意力,保留先验同时提升2D-3D对齐。基于初始对齐结果,计算重叠注意力偏置并直接注入另一预训练纹理生成模型,实现无需训练的纹理精准贴合。该设计使前馈分支学习基于生成先验的预测,生成分支则受前馈特征引导,互为增益。实验表明,本方法在DTU和BlendedMVS数据集上,3D对齐效果优于纯生成方法,姿态估计精度超越前馈方法。

原文摘要 · Abstract (English)

Recent trends in sparse-view 3D reconstruction have taken two different paths: feed-forward reconstruction that predicts pixel-aligned point maps without a complete geometry, and generative 3D reconstruction that generates complete geometry but often with poor input-alignment. We present Mix3R, a novel generative 3D reconstruction method which mixes feed-forward reconstruction and 3D generation into a single framework in an aligned manner. Mix3R generates a 3D shape in two stages: a sparse voxel generation stage and a textured geometry generation stage. Unlike pure generative methods, our first-stage generation jointly produces a coarse 3D structure (sparse voxels), per-view point maps and camera parameters aligned to that 3D structure. This is made possible by introducing a Mixture-of-Transformers architecture that inserts global self-attentions to a feed-forward reconstruction model and a 3D generative model, both pretrained on large-scale data. This design effectively retains the pretrained priors but enables better 2D-3D alignment. Based on the initial aligned generations of sparse 3D voxels and point maps, we compute an overlap-based attention bias that is directly added to another pretrained textured geometry generation model, enabling it to correctly place input textures onto generated shapes in a training-free manner. Our design brings mutual benefits to both feed-forward reconstruction and 3D generation: The feed-forward branch learns to ground its predictions to a generative 3D prior, and conversely, the 3D generation branch is conditioned on geometrically informative features from the feed-forward branch. As a result, our method produces 3D shapes with better input alignment compared with pure 3D generative methods, together with camera pose estimations more accurate than previous feed-forward reconstruction methods. Our project page is at https://jsnln.github.io/mix3r/

3D重建生成模型多视角对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。