提出DVC方法,动态评估数据价值以提升MLP训练效率。
Adaptive Data Selection for Multi-Layer Perceptron Training: A Sub-linear Value-Driven Method
- 按层分解数据贡献,结合质量、相关性与多样性多维评估
- 在六大数据集上优于八种基线,预算有限时精度和F1均更高
- 适合大规模异构数据训练,支持自适应样本选择
数据选择是神经网络训练的核心问题,尤其在多层感知机(MLP)中,如何从海量、多源、异构数据中,在预算限制下筛选最具价值的样本面临巨大挑战。现有方法如核子集构建、数据Shapley值和影响函数存在局限:过度简化非线性变换,忽略隐藏层中的有用表示,或因计算复杂度高难以扩展至大型MLP。为此,我们提出DVC(Data Value Contribution)方法,一种面向预算的新型数据评估与选择机制,考虑训练过程中网络参数的动态变化。DVC将数据贡献分解为层值贡献(LVC)与全局值贡献(GVC),设计六项指标及高效算法,从质量、相关性和分布多样性三个维度,多层次捕捉数据特征。DVC结合上限置信区间(UCB)算法实现自适应数据源选择,平衡探索与利用。在六个数据集和八种基线上的实验表明,该方法在不同预算条件下均持续超越现有方法,显著提升准确率与F1分数。本工作首次系统性地处理神经网络的分层数据评估,为大规模机器学习系统提供理论保障与实用优势。
原文摘要 · Abstract (English)
Data selection is one of the fundamental problems in neural network training, particularly for multi-layer perceptrons (MLPs) where identifying the most valuable training samples from massive, multi-source, and heterogeneous data sources under budget constraints poses significant challenges. Existing data selection methods, including coreset construction, data Shapley values, and influence functions, suffer from critical limitations: they oversimplify nonlinear transformations, ignore informative intermediate representations in hidden layers, or fail to scale to larger MLPs due to high computational complexity. In response, we propose DVC (Data Value Contribution), a novel budget-aware method for evaluating and selecting data for MLP training that accounts for the dynamic evolution of network parameters during training. The DVC method decomposes data contribution into Layer Value Contribution (LVC) and Global Value Contribution (GVC), employing six carefully designed metrics and corresponding efficient algorithms to capture data characteristics across three dimensions--quality, relevance, and distributional diversity--at different granularities. DVC integrates these assessments with an Upper Confidence Bound (UCB) algorithm for adaptive source selection that balances exploration and exploitation. Extensive experiments across six datasets and eight baselines demonstrate that our method consistently outperforms existing approaches under various budget constraints, achieving superior accuracy and F1 scores. Our approach represents the first systematic treatment of hierarchical data evaluation for neural networks, providing both theoretical guarantees and practical advantages for large-scale machine learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。