用符号与神经网络结合,让低质数据变优质,提升模型指令对齐效果。
ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix
- 符号模块按统计规律剔除噪声,神经模块用模型知识重建高质量样本
- 仅用低质数据训练的模型在5个基准上超越13种主流数据筛选方法
- 适合资源有限但想最大化利用原始数据的研究者和工程师
监督微调(SFT)通过在精心筛选的高质量指令-响应对子集上训练,将预训练大语言模型适配到特定领域。然而,现有以质量为先的方法常忽略被丢弃的低质数据中的潜在信号,且依赖不完美的质量过滤器。本文提出ENTP(Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix),通过符号净化与神经重建协同提升低质语料价值。符号模块基于统计先验识别并剔除噪声样本,神经组件则利用隐式表征与模型知识合成更丰富的指令-响应对。这种神经符号协作显著增强数据的信息量与多样性。实验表明,仅由低质数据构建的ENTP增强数据集,在五个指令遵循基准上优于13种现有数据选择基线,甚至超过在约30万例原数据上的微调效果。结果凸显了低质数据的未开发潜力,强调智能净化与合成对高效指令对齐的重要性。
原文摘要 · Abstract (English)
Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-response pairs, typically drawn from a larger dataset that often contains many low-quality or noisy samples. However, existing quality-first paradigms often overlook valuable signals in discarded low-quality data and rely on imperfect quality filters. We introduce ENTP (Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix), a framework that revitalizes low-quality corpora through symbolic purification and neural reconstruction. The symbolic module identifies and prunes noisy samples based on statistical priors, while the neural component synthesizes enriched instruction-response pairs by leveraging latent representations and model knowledge. This neural-symbolic synergy enhances data informativeness and diversity. Experiments show that ENTP-augmented datasets, constructed exclusively from low-quality data, outperform 13 established data-selection baselines across five instruction-following benchmarks, and even surpass fine-tuning on the full original dataset (approximately 300K examples). Our results highlight the untapped potential of low-quality data and underscore the importance of intelligent purification and synthesis for efficient instruction alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。