保护用户隐私的NeRF训练框架，防止数据泄露。
$S^2$NeRF: Privacy-preserving Training Framework for NeRF
- 采用分片学习分离数据与模型，避免本地数据上传。
- 在多个数据集上验证，可有效防御梯度泄露攻击。
- 适合需要保护敏感3D场景数据的应用场景。
神经辐射场（NeRF）革新了三维视觉与图形学，推动了虚拟现实和电子商务等领域的发展。然而，其对大量数据采集的依赖，尤其是包含敏感场景图像的数据，使用户在上传数据训练模型时面临严重隐私风险。为此，我们首先提出SplitNeRF，一种结合分片学习（SL）技术的训练框架，实现客户端与服务器间的隐私保护协作训练，无需共享本地数据。尽管如此，我们通过构建两种攻击方法——替代模型攻击与场景辅助替代模型攻击，揭示了SplitNeRF在梯度信息共享及少量泄露场景图像下的漏洞，可重建私有场景信息。为应对这些威胁，我们提出安全版SplitNeRF（S²NeRF），集成有效的防御机制。通过在共享梯度信息中引入与梯度范数相关的衰减噪声，S²NeRF在保障模型高实用性的同时显著提升隐私保护能力。我们在多个数据集上的广泛评估验证了S²NeRF对抗隐私泄露的有效性，证实其在敏感应用中的可行性。
原文摘要 · Abstract (English)
Neural Radiance Fields (NeRF) have revolutionized 3D computer vision and graphics, facilitating novel view synthesis and influencing sectors like extended reality and e-commerce. However, NeRF's dependence on extensive data collection, including sensitive scene image data, introduces significant privacy risks when users upload this data for model training. To address this concern, we first propose SplitNeRF, a training framework that incorporates split learning (SL) techniques to enable privacy-preserving collaborative model training between clients and servers without sharing local data. Despite its benefits, we identify vulnerabilities in SplitNeRF by developing two attack methods, Surrogate Model Attack and Scene-aided Surrogate Model Attack, which exploit the shared gradient data and a few leaked scene images to reconstruct private scene information. To counter these threats, we introduce $S^2$NeRF, secure SplitNeRF that integrates effective defense mechanisms. By introducing decaying noise related to the gradient norm into the shared gradient information, $S^2$NeRF preserves privacy while maintaining a high utility of the NeRF model. Our extensive evaluations across multiple datasets demonstrate the effectiveness of $S^2$NeRF against privacy breaches, confirming its viability for secure NeRF training in sensitive applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。