让大模型学会理解词语组合的整体意义,提升推理能力。
Improving Large Language Models with Concept-Aware Fine-Tuning
- 采用多标记联合预测机制,让模型学习完整概念而非逐字处理
- 在文本摘要和蛋白质设计任务中显著优于传统微调方法
- 首次实现后训练阶段的多标记预测,降低使用门槛
大语言模型(LLMs)已成为现代人工智能的核心。然而,现有的下一个词预测范式从根本上限制了模型形成连贯、高层级概念的能力,成为实现类人理解与推理的关键障碍。以‘ribonucleic acid’为例:模型会将其分解为多个词元(如“rib”、“on”等),依次学习,而非将整个短语视为统一的语义单元。这种碎片化表示阻碍了深层概念理解,最终制约真正智能系统的发展。为此,我们提出概念感知微调(CAFT),一种新型多标记训练方法,重新定义了大模型的微调方式。通过支持跨多个词元的序列学习,该方法促进更强的概念感知能力。实验表明,相比传统的下一个词微调方法,CAFT在文本摘要等通用任务以及从头设计蛋白质等专业任务中均有显著提升。多标记预测此前仅限于成本极高的预训练阶段;据我们所知,CAFT是首个将多标记设置引入后训练阶段的方法,从而有效使这一优势惠及更广泛的从业者与研究者。此外,该方法出乎意料的有效性也暗示其对机器学习研究社区具有更广泛的影响。所有代码与数据均开源于https://github.com/michaelchen-lab/caft-llm。
原文摘要 · Abstract (English)
Large language models (LLMs) have become the cornerstone of modern AI. However, the existing paradigm of next-token prediction fundamentally limits their ability to form coherent, high-level concepts, making it a critical barrier to human-like understanding and reasoning. Take the phrase "ribonucleic acid" as an example: an LLM will first decompose it into tokens, i.e., artificial text fragments ("rib", "on", ...), then learn each token sequentially, rather than grasping the phrase as a unified, coherent semantic entity. This fragmented representation hinders deeper conceptual understanding and, ultimately, the development of truly intelligent systems. In response, we introduce Concept-Aware Fine-Tuning (CAFT), a novel multi-token training method that redefines how LLMs are fine-tuned. By enabling the learning of sequences that span multiple tokens, this method fosters stronger concept-aware learning. Our experiments demonstrate significant improvements compared to conventional next-token finetuning methods across diverse tasks, including traditional applications like text summarization and domain-specific ones like de novo protein design. Multi-token prediction was previously only possible in the prohibitively expensive pretraining phase; CAFT, to our knowledge, is the first to bring the multi-token setting to the post-training phase, thus effectively democratizing its benefits for the broader community of practitioners and researchers. Finally, the unexpected effectiveness of our proposed method suggests wider implications for the machine learning research community. All code and data are available at https://github.com/michaelchen-lab/caft-llm
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。