量化3D生成模型记忆行为，发现数据多样性越高越易记忆
Memorization in 3D Shape Generation: An Empirical Study
- 设计评估框架量化3D生成模型的记忆程度
- 数据多样性与细粒度条件可显著提升记忆能力
- 通过长向量集和旋转增强可降低记忆且不损失质量
生成模型在3D视觉中被广泛用于合成新形状，但其生成是否依赖于记忆训练数据仍不明确。理解记忆行为有助于防止训练数据泄露并提升生成多样性。本文设计了一个评估框架，量化3D生成模型中的记忆现象，并研究不同数据与建模设计对记忆的影响。首先，将该框架应用于现有方法以量化记忆程度。随后，通过控制实验使用潜向量集（Vecset）扩散模型发现：在数据层面，记忆程度受数据模态影响，随数据多样性和更细粒度的条件输入而增加；在建模层面，记忆在中等引导尺度下达到峰值，可通过更长的Vecsets和简单旋转增强来缓解。整体框架与分析提供了对3D生成模型记忆现象的实证理解，并提出无需牺牲生成质量即可有效减少记忆的策略。代码已开源：https://github.com/zlab-princeton/3d_mem。
原文摘要 · Abstract (English)
Generative models are increasingly used in 3D vision to synthesize novel shapes, yet it remains unclear whether their generation relies on memorizing training shapes. Understanding their memorization could help prevent training data leakage and improve the diversity of generated results. In this paper, we design an evaluation framework to quantify memorization in 3D generative models and study the influence of different data and modeling designs on memorization. We first apply our framework to quantify memorization in existing methods. Next, through controlled experiments with a latent vector-set (Vecset) diffusion model, we find that, on the data side, memorization depends on data modality, and increases with data diversity and finer-grained conditioning; on the modeling side, it peaks at a moderate guidance scale and can be mitigated by longer Vecsets and simple rotation augmentation. Together, our framework and analysis provide an empirical understanding of memorization in 3D generative models and suggest simple yet effective strategies to reduce it without degrading generation quality. Our code is available at https://github.com/zlab-princeton/3d_mem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。