arXiv:2410.14254cs.LG2024-10

通过剔除冗余数据,用更小样本提升模型训练效率

RAZOR: Refining Accuracy by Zeroing Out Redundancies

  • 基于冗余检测筛选关键数据实例,减少无用信息
  • 在保持精度前提下,数据量可压缩至原规模的10%以下
  • 适用于有监督与无监督场景,适合大规模数据处理

在诸多应用领域,传感器和设备的激增带来了海量数据,给现有数据分析与挖掘技术带来巨大压力。然而,数据量增加并不等同于信息量提升,大量数据可能冗余或为噪声。这一问题在深度学习中尤为突出,额外数据的价值取决于其信息含量;否则只会增加计算开销与学习复杂度。为此,我们提出RAZOR——一种新型实例选择方法,能在不损害学习过程的前提下,从大规模数据集中提取出显著更小但足够具信息量的子集。RAZOR专为鲁棒性、高效性和可扩展性设计,适用于大规模数据。与多数已有方法不同,RAZOR可在有监督和无监督设置下运行。实验表明,相较于近期先进方法,RAZOR在有效性和效率方面均表现更优。

原文摘要 · Abstract (English)

In many application domains, the proliferation of sensors and devices is generating vast volumes of data, imposing significant pressure on existing data analysis and data mining techniques. Nevertheless, an increase in data volume does not inherently imply an increase in informational content, as a substantial portion may be redundant or represent noise. This challenge is particularly evident in the deep learning domain, where the utility of additional data is contingent on its informativeness. In the absence of such, larger datasets merely exacerbate the computational cost and complexity of the learning process. To address these challenges, we propose RAZOR, a novel instance selection technique designed to extract a significantly smaller yet sufficiently informative subset from a larger set of instances without compromising the learning process. RAZOR has been specifically engineered to be robust, efficient, and scalable, making it suitable for large-scale datasets. Unlike many techniques in the literature, RAZOR is capable of operating in both supervised and unsupervised settings. Experimental results demonstrate that RAZOR outperforms recent state-of-the-art techniques in terms of both effectiveness and efficiency.

数据压缩实例选择深度学习高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。