考虑计算预算的数据选择方法,让选数据更高效。
Computational Budget Should Be Considered in Data Selection
- 将计算预算纳入数据选择框架,构建双层优化模型。
- 在视觉与语言任务中,性能最高提升14.42%。
- 适合资源受限场景下的高效训练与模型部署。
数据选择通过选取有信息量的训练样本子集来提升计算效率。然而,现有方法忽视计算预算,将数据选择与重要性评估独立处理,导致在不同预算下无一算法能稳定优于其他方法(甚至随机选择)。我们提出计算预算感知的数据选择(CADS)方法,将其自然建模为双层优化问题:内层在给定计算预算下对所选数据子集进行模型训练,外层基于模型表现优化数据选择。技术上解决两大挑战:外层梯度计算中昂贵的海森矩阵估计,以及内层迭代中达到最优解的计算负担。针对前者,提出概率重参数化策略,使用无海森矩阵的策略梯度估计器;针对后者,将内层优化转化为外层目标中的惩罚项,并发现仅需估计一维损失最小值即可计算梯度,显著提升效率。大量实验表明,该方法在视觉与语言基准上相较基线最高提升14.42%。
原文摘要 · Abstract (English)
Data selection improves computational efficiency by choosing informative subsets of training samples. However, existing methods ignore the compute budget, treating data selection and importance evaluation independently of compute budget constraints. Yet empirical studies show no algorithm can consistently outperform others (or even random selection) across varying budgets. We therefore argue that compute budget must be integral to data-selection strategies, since different budgets impose distinct requirements on data quantity, quality, and distribution for effective training. To this end, we propose a novel Computational budget-Aware Data Selection (CADS) method and naturally formulate it into a bilevel optimization framework, where the inner loop trains the model within the constraints of the computational budget on some selected subset of training data, while the outer loop optimizes data selection based on model evaluation. Our technical contributions lie in addressing two main challenges in solving this bilevel optimization problem: the expensive Hessian matrix estimation for outer-loop gradients and the computational burden of achieving inner-loop optimality during iterations. To solve the first issue, we propose a probabilistic reparameterization strategy and compute the gradient using a Hessian-free policy gradient estimator. To address the second challenge, we transform the inner optimization problem into a penalty term in the outer objective, further discovering that we only need to estimate the minimum of a one-dimensional loss to calculate the gradient, significantly improving efficiency. Extensive experiments show that our method achieves performance gains of up to 14.42% over baselines in vision and language benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。