用剪枝+融合加速数据选择，降低训练成本。
Pruning-based Data Selection and Network Fusion for Efficient Deep Learning
- 先剪枝生成小模型选关键数据，再融合到原模型。
- 数据选择计算量减少，训练速度提升明显。
- 适合资源有限场景下的高效训练优化。
高效的数据选择对提升深度神经网络训练效率、降低标注成本至关重要。然而，传统方法通常计算开销大，限制了其可扩展性和实际应用。我们提出 PruneFuse，一种结合剪枝与网络融合的新方法，用于增强数据选择并加速网络训练。在 PruneFuse 中，原始密集网络被剪枝以生成更小的代理模型，该模型能高效地从数据集中筛选出最具信息量的样本。当迭代数据选择达到足够样本后，剪枝模型所学得的知识通过网络融合无缝整合至密集模型中，提供优化的初始化，从而加速训练过程。在多个数据集上的大量实验表明，PruneFuse 显著降低了数据选择的计算成本，性能优于基线方法，并加快了整体训练进程。
原文摘要 · Abstract (English)
Efficient data selection is essential for improving the training efficiency of deep neural networks and reducing the associated annotation costs. However, traditional methods tend to be computationally expensive, limiting their scalability and real-world applicability. We introduce PruneFuse, a novel method that combines pruning and network fusion to enhance data selection and accelerate network training. In PruneFuse, the original dense network is pruned to generate a smaller surrogate model that efficiently selects the most informative samples from the dataset. Once this iterative data selection selects sufficient samples, the insights learned from the pruned model are seamlessly integrated with the dense model through network fusion, providing an optimized initialization that accelerates training. Extensive experimentation on various datasets demonstrates that PruneFuse significantly reduces computational costs for data selection, achieves better performance than baselines, and accelerates the overall training process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。