一分钟内从单张图片生成可交互的3D场景，无需优化
FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minute
- 端到端网络直接从RGB图像预测物体位姿、网格和纹理
- 生成完整且物理解耦的3D场景，速度比现有方法快数个数量级
- 适合游戏开发与交互应用，无需额外训练或调参
我们提出FIRE3D，一个统一框架，仅需单张RGB图像或随意拍摄的视频，在一分钟内生成可用于游戏与交互应用的仿真级3D场景资产。FIRE3D的核心是一个前馈式端到端网络，基于从RGB输入估算的带姿态的RGB-D观测，预测出包含6-DoF位姿、边界框、网格和纹理的组合式场景表示。通过将场景建模为离散实体集合，FIRE3D生成了模态完整的、可直接交互的环境，其中物体相互独立且具备物理属性。该框架无需测试时优化，运行速度比以往交互式方法快数个数量级，且在物体级完整性上超越现有前馈式3D方法。我们在多个数据集上验证了其在位姿精度、几何完整性与纹理质量上的竞争力甚至达到当前最优水平，同时保持显著更快的速度。
原文摘要 · Abstract (English)
We present FIRE3D, a unified framework that takes a single RGB image or casual RGB video and transforms it into simulation-ready 3D scene assets for games and interactive applications in under a minute. At the core of FIRE3D is a feed-forward, end-to-end network that predicts a compositional scene representation from posed RGB-D observations estimated from the RGB capture, including the 6-DoF pose, bounding box, mesh, and texture for every object. By modeling the scene as a collection of discrete entities, FIRE3D produces amodally complete and simulation-ready environments where objects are physically decoupled and ready for interaction. Our framework requires no test-time optimization, runs orders of magnitude faster than prior interaction-ready methods, and provides object-level completeness beyond existing feed-forward 3D approaches. We demonstrate competitive or state-of-the-art results across pose accuracy, geometry completeness, and texture quality across various datasets while being orders of magnitudes faster. Project page: https://xiahongchi.github.io/Fire3D/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。