用神经场压缩数据集，高效保留训练信息
Distilling Dataset into Neural Field
- 用神经场参数化合成数据，以坐标为输入生成多样内容
- 在相同预算下，表达能力优于已有方法，跨图像/视频/音频/3D表现更优
- 适合需要降低数据存储与计算成本的研究者
大规模数据集对训练高性能深度学习模型至关重要，但带来显著的计算与存储开销。为应对这一挑战，数据集蒸馏通过将大规模数据压缩为更小的合成数据集，保留训练所需的关键信息，成为有前景的解决方案。本文提出一种新型参数化框架——数据集蒸馏到神经场（DDiF），利用神经场存储大规模数据集的核心信息。由于神经场以坐标为输入、输出量值的独特性质，DDiF能有效保留信息，并可轻松生成多种形状的数据。理论证明，在相同单个合成实例预算下，DDiF的表达能力优于部分已有方法。大量实验表明，DDiF在多个基准数据集上表现优异，应用范围扩展至图像、视频、音频及3D体素领域。代码已开源：https://github.com/aailab-kaist/DDiF。
原文摘要 · Abstract (English)
Utilizing a large-scale dataset is essential for training high-performance deep learning models, but it also comes with substantial computation and storage costs. To overcome these challenges, dataset distillation has emerged as a promising solution by compressing the large-scale dataset into a smaller synthetic dataset that retains the essential information needed for training. This paper proposes a novel parameterization framework for dataset distillation, coined Distilling Dataset into Neural Field (DDiF), which leverages the neural field to store the necessary information of the large-scale dataset. Due to the unique nature of the neural field, which takes coordinates as input and output quantity, DDiF effectively preserves the information and easily generates various shapes of data. We theoretically confirm that DDiF exhibits greater expressiveness than some previous literature when the utilized budget for a single synthetic instance is the same. Through extensive experiments, we demonstrate that DDiF achieves superior performance on several benchmark datasets, extending beyond the image domain to include video, audio, and 3D voxel. We release the code at https://github.com/aailab-kaist/DDiF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。