揭示大模型内在自纠错的隐藏缺陷，提出改进方法。
Understanding the Dark Side of LLMs' Intrinsic Self-Correction
- 通过三类任务分析自纠错机制，发现其易引发答案波动和提示偏见。
- 在复杂任务中引入类人类认知偏差，导致错误持续放大。
- 提出重复提问与少量样本微调策略，有效缓解问题，适合模型优化者参考。
内在自纠错旨在仅依赖大模型自身能力，通过反馈提示提升输出质量。然而，近期研究发现，若缺乏真实标签作为反馈，该机制会失效。本文针对不同任务，深入解析大模型内在自纠错的表现，涵盖简单事实问答及三类复杂任务，使用ChatGPT系列（o1, 4o, 3.5-turbo）与Llama系列（2-7B, 3-8B, 3.1-8B）等主流模型，设计三种解释方法。研究发现：（1）内在自纠错会导致模型在中间与最终答案间反复摇摆，并在简单事实问题上产生提示偏见；（2）在复杂任务中引入类人类认知偏差。基于此，提出两种简单有效的缓解策略：问题重复与少量样本监督微调。相关代码与数据已开源：https://x-isc.info/。
原文摘要 · Abstract (English)
Intrinsic self-correction was proposed to improve LLMs' responses via feedback prompts solely based on their inherent capability. However, recent works show that LLMs' intrinsic self-correction fails without oracle labels as feedback prompts. In this paper, we aim to interpret LLMs' intrinsic self-correction for different tasks, especially for those failure cases. By including one simple task and three complex tasks with state-of-the-art (SOTA) LLMs like ChatGPT families (o1, 4o, 3.5-turbo) and Llama families (2-7B, 3-8B, and 3.1-8B), we design three interpretation methods to reveal the dark side of LLMs' intrinsic self-correction. We identify intrinsic self-correction can (1) cause LLMs to waver both intermedia and final answers and lead to prompt bias on simple factual questions; (2) introduce human-like cognitive bias on complex tasks. In light of our findings, we also provide two simple yet effective strategies for alleviation: question repeating and supervised fine-tuning with a few samples. We open-source our work at https://x-isc.info/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。