攻击3D重建系统的初始阶段,让多个系统生成低质模型。
PoInit-of-View: Poisoning Initialization of Views Transfers Across Multiple 3D Reconstruction Systems

- 针对结构光初始化模块,制造跨视角梯度不一致
- 黑盒攻击下PSNR提升25.1%,SSIM提升16.5%
- 适用于多种3D重建系统,可实现迁移攻击
现有研究对3D重建系统输入视图的投毒攻击多采用端到端反向传播,未揭示重建流程中特定模块的新漏洞。本文指出,作为众多主流重建系统几何核心的结构从运动(SfM)初始化阶段,可被针对性攻击以实现跨系统转移。为此,我们提出PoInit-of-View,通过优化对抗扰动,在对应3D点的投影中引入跨视角梯度不一致,破坏关键点检测与特征匹配,进而干扰SfM中的位姿估计与三角化,最终导致渲染视图质量下降。理论分析揭示了跨视角不一致性与对应关系崩溃的关联。实验表明,该方法在多种3D重建系统与数据集上有效,在黑盒迁移场景(如3DGS到NeRF)中,相比单视图基线,PSNR提升25.1%,SSIM提升16.5%。
原文摘要 · Abstract (English)
Poisoning input views of 3D reconstruction systems has been recently studied. However, we identify that existing studies simply backpropagate adversarial gradients through the 3D reconstruction pipeline as a whole, without uncovering the new vulnerability rooted in specific modules of the 3D reconstruction pipeline. In this paper, we argue that the structure-from-motion (SfM) initialization, as the geometric core of many widely used reconstruction systems, can be targeted to achieve transferable poisoning effects across diverse 3D reconstruction systems. To this end, we propose PoInit-of-View, which optimizes adversarial perturbations to intentionally introduce cross-view gradient inconsistencies at projections of corresponding 3D points. These inconsistencies disrupt keypoint detection and feature matching, thereby corrupting pose estimation and triangulation within SfM, eventually resulting in low-quality rendered views. We also provide a theoretical analysis that connects cross-view inconsistency to correspondence collapse. Experimental results demonstrate the effectiveness of our PoInit-of-View on diverse 3D reconstruction systems and datasets, surpassing the single-view baseline by 25.1% in PSNR and 16.5% in SSIM in black-box transfer settings, such as 3DGS to NeRF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。