arXiv:2510.23160cs.CL2025-10被引 2

用符号与神经网络结合,让低质数据变优质,提升模型指令对齐效果。

ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix

  • 符号模块按统计规律剔除噪声,神经模块用模型知识重建高质量样本
  • 仅用低质数据训练的模型在5个基准上超越13种主流数据筛选方法
  • 适合资源有限但想最大化利用原始数据的研究者和工程师

监督微调(SFT)通过在精心筛选的高质量指令-响应对子集上训练,将预训练大语言模型适配到特定领域。然而,现有以质量为先的方法常忽略被丢弃的低质数据中的潜在信号,且依赖不完美的质量过滤器。本文提出ENTP(Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix),通过符号净化与神经重建协同提升低质语料价值。符号模块基于统计先验识别并剔除噪声样本,神经组件则利用隐式表征与模型知识合成更丰富的指令-响应对。这种神经符号协作显著增强数据的信息量与多样性。实验表明,仅由低质数据构建的ENTP增强数据集,在五个指令遵循基准上优于13种现有数据选择基线,甚至超过在约30万例原数据上的微调效果。结果凸显了低质数据的未开发潜力,强调智能净化与合成对高效指令对齐的重要性。

原文摘要 · Abstract (English)

Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-response pairs, typically drawn from a larger dataset that often contains many low-quality or noisy samples. However, existing quality-first paradigms often overlook valuable signals in discarded low-quality data and rely on imperfect quality filters. We introduce ENTP (Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix), a framework that revitalizes low-quality corpora through symbolic purification and neural reconstruction. The symbolic module identifies and prunes noisy samples based on statistical priors, while the neural component synthesizes enriched instruction-response pairs by leveraging latent representations and model knowledge. This neural-symbolic synergy enhances data informativeness and diversity. Experiments show that ENTP-augmented datasets, constructed exclusively from low-quality data, outperform 13 established data-selection baselines across five instruction-following benchmarks, and even surpass fine-tuning on the full original dataset (approximately 300K examples). Our results highlight the untapped potential of low-quality data and underscore the importance of intelligent purification and synthesis for efficient instruction alignment.

指令微调数据增强神经符号系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。