用梯度感知动态选样本，训练更快更省电。
GRAFT: Gradient-Aware Fast MaxVol Technique for Dynamic Data Sampling
- 在低秩子空间中选代表性样本，减少计算量。
- 相比全批量训练，准确率相当但耗时降低40%以上。
- 适合追求高效低碳训练的研究者和工程师。
在大规模数据集上训练现代神经网络具有高昂的计算与环境成本。我们提出GRAFT，一种可扩展的训练中子集选择方法：(i) 对每个批次提取低秩特征表示；(ii) 使用快速MaxVol采样器选取能覆盖批次主要子空间的小规模多样化子集；(iii) 基于梯度近似准则动态调整子集大小。通过在低秩子空间中操作并仅对精选样本进行训练，GRAFT 在保持训练轨迹的同时显著降低运行时间、能耗和二氧化碳排放。在多个基准测试中，GRAFT 在准确率和效率方面均达到或超越现有选择方法，实现了准确率、效率与碳排放之间的良好平衡。
原文摘要 · Abstract (English)
Training modern neural networks on large datasets is computationally and environmentally costly. We introduce GRAFT, a scalable in-training subset selection method that (i) extracts a low-rank feature representation for each batch, (ii) applies a Fast MaxVol sampler to select a small, diverse subset that spans the batch's dominant subspace, and (iii) dynamically adjusts the subset size using a gradient-approximation criterion. By operating in low-rank subspaces and training on carefully chosen examples instead of full batches, GRAFT preserves the training trajectory while reducing wall-clock time, energy consumption, and $\mathrm{CO}_2$ emissions. Across multiple benchmarks, GRAFT matches or exceeds recent selection baselines in both accuracy and efficiency, providing a favorable trade-off between accuracy, efficiency, and emissions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。