梳理前馈3D重建的五大核心设计问题,助力高效通用建模
Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective

- 按模型设计策略而非输出形式分类,提炼共性架构模式
- 提出五类关键问题:特征增强、几何感知、效率优化等
- 适合关注3D建模系统设计与评估的研究者参考
从2D输入重建3D表示是计算机视觉与图形学的基础任务,传统方法虽精度高但存在单场景优化慢或类别特定训练的局限,制约了实际部署与可扩展性。近年来,通用前馈3D重建快速发展——通过单次前向传播将图像映射为3D表示,实现高效重建与跨场景鲁棒泛化。本综述观察到:尽管输出形式多样(如隐式场、显式图元),现有前馈方法共享相似高层架构模式,包括图像特征提取骨干、多视角信息融合机制和几何感知设计原则。因此,本文脱离输出差异,聚焦模型设计,提出一种与输出格式无关的新分类体系,将研究方向归纳为五个驱动性问题:特征增强、几何感知、模型效率、增广策略与时序感知建模。为支撑该分类体系,我们系统回顾相关基准与数据集,全面讨论并分类基于前馈3D模型的实际应用场景。最后,展望未来挑战,如可扩展性、评估标准与世界建模。
原文摘要 · Abstract (English)
Reconstructing 3D representations from 2D inputs is a fundamental task in computer vision and graphics, serving as a cornerstone for understanding and interacting with the physical world. While traditional methods achieve high fidelity, they are limited by slow per-scene optimization or category-specific training, which hinders their practical deployment and scalability. Hence, generalizable feed-forward 3D reconstruction has witnessed rapid development in recent years. By learning a model that maps images directly to 3D representations in a single forward pass, these methods enable efficient reconstruction and robust cross-scene generalization. Our survey is motivated by a critical observation: despite the diverse geometric output representations, ranging from implicit fields to explicit primitives, existing feed-forward approaches share similar high-level architectural patterns, such as image feature extraction backbones, multi-view information fusion mechanisms, and geometry-aware design principles. Consequently, we abstract away from these representation differences and instead focus on model design, proposing a novel taxonomy centered on model design strategies that are agnostic to the output format. Our proposed taxonomy organizes the research directions into five key problems that drive recent research development: feature enhancement, geometry awareness, model efficiency, augmentation strategies and temporal-aware models. To support this taxonomy with empirical grounding and standardized evaluation, we further comprehensively review related benchmarks and datasets, and extensively discuss and categorize real-world applications based on feed-forward 3D models. Finally, we outline future directions to address open challenges such as scalability, evaluation standards, and world modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。