arXiv:2506.07169cs.CLcs.AI2025-06

提出新方法减少文本分类训练数据量,节省算力且保持效果。

CTDGSI: A comprehensive exploitation of instance selection methods for automatic text classification. VII Concurso de Teses, Dissertações e Trabalhos de Graduação em SI -- XXI Simpósio Brasileiro de Sistemas de Informação

  • 设计噪声敏感与冗余感知的实例选择算法
  • 平均减少41%训练数据,速度提升1.67倍(最高2.46倍)
  • 适合大规模文本分类任务,尤其适配Transformer模型

自然语言处理的发展依赖于更多数据、更强算力和更复杂模型,典型代表为大语言模型。然而,针对特定应用训练(或微调)大型密集模型通常需要大量计算资源。本文聚焦于尚未充分研究的NLP数据工程技术——实例选择(Instance Selection, IS),其目标是在保留模型性能的同时,通过剔除噪声或冗余样本减少训练集规模,从而降低训练成本。本文对自动文本分类(ATC)任务中多种IS方法进行了全面、科学的比较,涵盖多个分类模型和数据集。研究发现IS存在巨大未开发潜力。为此,本文提出两种新型IS方法,分别侧重噪声识别与冗余感知,专为大规模数据集和Transformer架构设计。最终方案在所有数据集上实现平均41%的训练集缩减,同时保持原有模型性能,并带来1.67倍(最高达2.46倍)的速度提升,具备处理数十万文档数据集的可扩展性。

原文摘要 · Abstract (English)

Progress in Natural Language Processing (NLP) has been dictated by the rule of more: more data, more computing power and more complexity, best exemplified by the Large Language Models. However, training (or fine-tuning) large dense models for specific applications usually requires significant amounts of computing resources. This \textbf{Ph.D. dissertation} focuses on an under-investi\-gated NLP data engineering technique, whose potential is enormous in the current scenario known as Instance Selection (IS). The IS goal is to reduce the training set size by removing noisy or redundant instances while maintaining the effectiveness of the trained models and reducing the training process cost. We provide a comprehensive and scientifically sound comparison of IS methods applied to an essential NLP task -- Automatic Text Classification (ATC), considering several classification solutions and many datasets. Our findings reveal a significant untapped potential for IS solutions. We also propose two novel IS solutions that are noise-oriented and redundancy-aware, specifically designed for large datasets and transformer architectures. Our final solution achieved an average reduction of 41\% in training sets, while maintaining the same levels of effectiveness in all datasets. Importantly, our solutions demonstrated speedup improvements of 1.67x (up to 2.46x), making them scalable for datasets with hundreds of thousands of documents.

文本分类实例选择Transformer数据压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。