用单图生成无限视频深度数据，提升3D重建精度
Puzzles: Unbounded Video-Depth Augmentation for Scalable End-to-End 3D Reconstruction
- 通过图像变换模拟多样相机轨迹和真实场景几何
- 仅用10%原始数据+增强后性能媲美全量数据训练
- 无需改动模型结构，适配现有3D重建系统
多视角3D重建仍是计算机视觉核心挑战。近期方法如DUST3R可直接从图像对回归点云，无需已知场景几何或相机参数，但其性能受限于训练数据的多样性与规模。本文提出Puzzles，一种数据增强策略，可从单张图像或视频片段合成无限高质量带位姿视频-深度数据。通过针对性图像变换模拟多样相机轨迹与真实场景几何，显著提升数据多样性。大量实验表明，将Puzzles集成至现有基于视频的3D重建流程中，可持续提升性能，且不需修改网络架构。值得注意的是，仅使用原始数据10%并经Puzzles增强后，模型性能即可媲美全量数据训练结果。代码已公开于https://jiahao-ma.github.io/puzzles/。
原文摘要 · Abstract (English)
Multi-view 3D reconstruction remains a core challenge in computer vision. Recent methods, such as DUST3R and its successors, directly regress pointmaps from image pairs without relying on known scene geometry or camera parameters. However, the performance of these models is constrained by the diversity and scale of available training data. In this work, we introduce Puzzles, a data augmentation strategy that synthesizes an unbounded volume of high-quality posed video-depth data from a single image or video clip. By simulating diverse camera trajectories and realistic scene geometry through targeted image transformations, Puzzles significantly enhances data variety. Extensive experiments show that integrating Puzzles into existing video-based 3D reconstruction pipelines consistently boosts performance without modifying the underlying network architecture. Notably, models trained on only ten percent of the original data augmented with Puzzles still achieve accuracy comparable to those trained on the full dataset. Code is available at https://jiahao-ma.github.io/puzzles/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。