用代码逻辑训练让大模型减少幻觉，通用性强
MIH-TCCT: Mitigating Inconsistent Hallucinations in LLMs via Event-Driven Text-Code Cyclic Training
- 通过事件驱动生成代码，循环训练传递逻辑一致性
- 在3个主流大模型上显著降低幻觉率，保持性能
- 无需适配下游任务，适合各类自然语言任务
近期利用合成数据集的方法试图解决大语言模型（LLMs）中的不一致幻觉问题，但这些方法多针对特定任务，泛化能力有限。受代码训练模型在逻辑密集型领域表现优异的启发，我们提出一种新框架：基于事件驱动生成对应代码，并通过循环训练将代码的逻辑一致性有效迁移至自然语言。该方法在三个领先大模型和两类自然语言任务中显著减少了不一致幻觉，同时保持整体性能。该框架无需适配下游任务，展现出良好的通用性，为解决不一致幻觉提供了新思路。
原文摘要 · Abstract (English)
Recent methodologies utilizing synthetic datasets have aimed to address inconsistent hallucinations in large language models (LLMs); however,these approaches are primarily tailored to specific tasks, limiting their generalizability. Inspired by the strong performance of code-trained models in logic-intensive domains, we propose a novel framework that leverages event-based text to generate corresponding code and employs cyclic training to transfer the logical consistency of code to natural language effectively. Our method significantly reduces inconsistent hallucinations across three leading LLMs and two categories of natural language tasks while maintaining overall performance. This framework effectively alleviates hallucinations without necessitating adaptation to downstream tasks, demonstrating generality and providing new perspectives to tackle the challenge of inconsistent hallucinations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。