用博弈论游戏提升大模型输出的一致性与可靠性
Truth or Deceit? A Bayesian Decoding Game Enhances Consistency and Reliability
- 将解码过程设计为多阶段贝叶斯博弈，动态收敛到可靠输出
- 小模型（78.1）可超越大模型（76.6），无需人工反馈
- 适合追求高可信度输出的场景，如医疗、法律等关键领域
大语言模型在复杂或模糊场景下常产生看似合理但缺乏一致性和可靠性的输出。现有方法往往以牺牲准确性为代价换取一致性。为此，我们提出一种新颖的博弈论解码方法，将解码过程建模为多阶段贝叶斯解码博弈，通过正确性对齐确保一致性，通过模糊性校准提升可靠性。该方法使模型在无须人工反馈或额外训练的情况下，动态收敛至最可靠的输出，并区分有效与伪效输出。实验表明，该机制使较小模型（LLaMA-13B，78.1）在性能上超越更大模型（PaLM-540B，76.6），并能整合多种解码策略与模型，证明了博弈论工具在提升大模型真实性与可靠性方面的潜力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often produce outputs that -- though plausible -- can lack consistency and reliability, particularly in ambiguous or complex scenarios. Challenges arise from ensuring that outputs align with both factual correctness and human intent. This is problematic in existing approaches that trade improved consistency for lower accuracy. To mitigate these challenges, we propose a novel game-theoretic approach to enhance consistency and reliability during the decoding stage of LLM output generation. Our method models the decoding process as a multistage Bayesian decoding game. This ensures consistency through Correctness Alignment and enhances reliability via Ambiguity Calibration. The model dynamically converges to a consensus on the most reliable outputs and distinguishes {Valid, Specious} outputs without human feedback or additional training. Our game design allows smaller models to outperform much larger models through game mechanisms (e.g., 78.1 LLaMA13B vs 76.6 PaLM540B), as well as integrating various LL strategies and models, demonstrating the potential of game-theoretic tools to improve the truthfulness and reliability of LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。