用潜在概念专家混合机制,让单个大模型自动生成多角度推理,对抗提示偏见。
Single LLM Debate, MoLaCE: Mixture of Latent Concept Experts Against Confirmation Bias
- 通过不同激活强度的潜在概念专家混合,动态响应提示语义
- 在多个数据集上显著降低确认偏见,提升推理鲁棒性
- 适合需要高效、可靠推理的场景,如单模型智能体系统
大型语言模型极易受输入确认偏见影响:当提示暗示特定答案时,模型常强化该倾向而非探索其他可能。这一现象虽未被充分研究,却已在基础模型中造成危害,并在多智能体辩论中加剧,导致回音室效应而非纠错。我们提出一种轻量级推理时框架 MoLaCE(潜在概念专家混合),通过在塑造模型输出的潜在概念上以不同激活强度组合专家,缓解确认偏见。关键洞察在于,由于语言的组合特性,不同表述的提示会以特定方式重加权潜在概念,从而影响事实准确性,因此无法对所有输入采用单一固定干预。该设计使单个大模型内生实现辩论效果,兼具计算效率与可扩展性,亦可嵌入多智能体辩论框架以增强视角多样性并减少相关错误。实证表明,该方法持续降低确认偏见,提升鲁棒性,性能可媲美甚至超越多智能体辩论,同时仅需极小计算开销。
原文摘要 · Abstract (English)
Large language models (LLMs) are highly vulnerable to input confirmation bias. When a prompt implies a preferred answer, models often reinforce that bias rather than explore alternatives. This phenomenon remains underexplored, yet it is already harmful in base models and poses an even greater risk in multi-agent debate, where echo chambers reinforce bias instead of correction. We introduce Mixture of Latent Concept Experts (MoLaCE), a lightweight inference-time framework that addresses confirmation bias by mixing experts instantiated as different activation strengths over latent concepts that shape model responses. Our key insight is that, due to the compositional nature of language, differently phrased prompts reweight latent concepts in prompt-specific ways that affect factual correctness, so no single fixed intervention can be applied universally across inputs. This design enables a single LLM to emulate the benefits of debate internally while remaining computationally efficient and scalable. It can also be integrated into multi-agent debate frameworks to diversify perspectives and reduce correlated errors. We empirically show that it consistently reduces confirmation bias, improves robustness, and matches or surpasses multi-agent debate while requiring only a fraction of the computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。