对比两种数据清洗方法，发现小数据高噪声时效果显著。
Data filtering methods for training language models
- 用置信学习和数据制图法检测标签错误
- 小数据集上置信学习提升F1-macro，大样本无效
- 针对性清洗优于随机删除，适合低质量小数据
数据质量是机器学习模型有效性的关键因素。标签错误即使在常用基准中也存在，会引入训练数据噪声并降低模型泛化能力。本文对两种自动标签错误检测方法——置信学习与数据制图法——在三个不同规模、类别数和领域的俄语文本分类语料库上进行对比分析：ru_emotion_e-culture（49,123条，情感分类）、RuCoLA（8,524条，语言可接受性）和TERRa（2,337条，文本蕴含识别）。采用预训练的rubert-base-cased模型在各语料库上微调。为验证过滤的有效性，进行了等量随机删除的对照实验。结果表明，两种方法的效果高度依赖数据集特性：在大规模低噪声语料上过滤无提升；而在小规模高噪声语料上，置信学习实现显著的F1-macro提升。数据制图法表现更保守，删减样本更少。所有语料上，针对性删除均优于随机删除，证实了方法的有效性。
原文摘要 · Abstract (English)
Data quality is a critical factor in the effectiveness of machine learning models. Label errors, present even in widely used benchmarks, introduce noise into training data and reduce model generalization. In this work, we conduct a comparative analysis of two automatic label error detection methods - Confident Learning and Dataset Cartography - on three Russian text classification corpora of varying size, number of classes, and domain: ru_emotion_e-culture (49,123 examples, emotion classification), RuCoLA (8,524 examples, linguistic acceptability), and TERRa (2,337 examples, textual entailment recognition). We use the pre-trained rubert-base-cased model fine-tuned on each corpus. To verify the meaningfulness of filtering, we conduct control experiments with random removal of an equivalent number of examples. Results show that the effectiveness of both methods depends strongly on dataset characteristics: on large corpora with low noise levels, filtering does not improve performance, while on small datasets with high noise, Confident Learning achieves a significant F1-macro improvement. Dataset Cartography demonstrates more conservative behavior, removing fewer examples. Across all corpora, targeted removal by both methods outperforms random removal, confirming the meaningfulness of the approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。