arXiv:2509.06524cs.CL2025-09AAAI被引 3

用大模型自身做领域数据筛选,高效又精准。

LAMDAS: LLM as an Implicit Classifier for Domain-specific Data Selection

  • 让预训练大模型充当隐式分类器,自动识别目标领域数据
  • 仅用少量参考数据就能选出高质候选集,性能超越全量数据训练
  • 兼顾效果与效率,适合资源受限的领域适配场景

将大语言模型(LLM)适配到特定领域常受限于高质量人工标注数据稀缺。尽管大量未经审核的数据易获取,但盲目使用会引入噪声并降低性能。因此,高效准确的数据选择至关重要。现有方法分为基于相似性的和直接优化类,难以兼顾二者。本文提出LAMDAS(LLM As an iMplicit classifier for domain-specific DAta Selection),利用预训练LLM自身作为隐式分类器,避免显式特征工程和耗时优化。将数据选择重构为一类别分类问题,通过小规模参考数据识别属于目标领域的候选数据。大量实验表明,LAMDAS仅用极少数据即超越全量数据训练,且在多种场景下优于九个当前最优(SOTA)基线。同时,在性能提升与计算效率之间达到最优平衡。

原文摘要 · Abstract (English)

Adapting large language models (LLMs) to specific domains often faces a critical bottleneck: the scarcity of high-quality, human-curated data. While large volumes of unchecked data are readily available, indiscriminately using them for fine-tuning risks introducing noise and degrading performance. Strategic data selection is thus crucial, requiring a method that is both accurate and efficient. Existing approaches, categorized as similarity-based and direct optimization methods, struggle to simultaneously achieve these goals. In this paper, we introduce LAMDAS (LLM As an iMplicit classifier for domain-specific DAta Selection), a novel approach that leverages the pre-trained LLM itself as an implicit classifier, thereby bypassing explicit feature engineering and computationally intensive optimization process. LAMDAS reframes data selection as a one-class classification problem, identifying candidate data that "belongs" to the target domain defined by a small reference dataset. Extensive experimental results demonstrate that LAMDAS not only exceeds the performance of full-data training using a fraction of the data but also outperforms nine state-of-the-art (SOTA) baselines under various scenarios. Furthermore, LAMDAS achieves the most compelling balance between performance gains and computational efficiency compared to all evaluated baselines.

数据筛选大模型领域适应高效学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。