通过注入词语推理过程，提升大模型的逻辑推理能力与准确性。
ToW: Thoughts of Words Improve Reasoning in Large Language Models
- 在预训练中引入细粒度词语推理注释，增强模型理解上下文的能力。
- 仅用7万条注释持续预训练，推理能力平均提升7%~9%，幻觉减少10%。
- 方法通用无偏见，适用于各类任务，无需额外标签设计。
我们提出词语推理（ToW），一种新的训练阶段数据增强方法，用于下一步词预测。ToW将下一步词预测视为核心推理任务，在预训练文本中注入解释下一个词应为何及其与前文关系的细粒度推理过程。该方法解决了现有下一词预测学习方案的两个根本缺陷：易导致事实性幻觉，且难以让模型有效学习原始文本中的隐含推理过程。尽管获取这些词语推理的方式多样，我们首次探索通过大模型蒸馏来生成ToW标注。仅使用7万条ToW标注进行持续预训练后，模型在推理性能上平均提升7%至9%，幻觉现象最多降低10%。同时，ToW完全与任务和应用无关，不引入任何标签或语义偏差。
原文摘要 · Abstract (English)
We introduce thoughts of words (ToW), a novel training-time data-augmentation method for next-word prediction. ToW views next-word prediction as a core reasoning task and injects fine-grained thoughts explaining what the next word should be and how it is related to the previous contexts in pre-training texts. Our formulation addresses two fundamental drawbacks of existing next-word prediction learning schemes: they induce factual hallucination and are inefficient for models to learn the implicit reasoning processes in raw texts. While there are many ways to acquire such thoughts of words, we explore the first step of acquiring ToW annotations through distilling from larger models. After continual pre-training with only 70K ToW annotations, we effectively improve models' reasoning performances by 7% to 9% on average and reduce model hallucination by up to 10%. At the same time, ToW is entirely agnostic to tasks and applications, introducing no additional biases on labels or semantics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。