发现大模型会受初始信息误导，且简单方法难纠正。
Anchoring Bias in Large Language Models: An Experimental Study
- 用实验数据测试大模型对初始信息的依赖性
- 仅靠提示词改写无法有效缓解锚定偏差
- 需多角度收集信息才能减少模型偏差
大型语言模型（如GPT-4和Gemini）在生成与理解人类文本方面取得显著进展。尽管能力强大，它们仍存在各类偏见，其中认知偏见研究不足。本研究聚焦锚定偏差——即初始信息过度影响判断的现象。通过实验数据集，我们检验了锚定偏差在大模型中的表现，并评估多种缓解策略的有效性。结果表明，大模型响应极易受有偏提示影响；仅靠链式思维、原则思考、忽略提示或反思等简单算法无法有效缓解偏差。真正有效的策略需从多角度收集信息，避免模型被单一信息锚定。
原文摘要 · Abstract (English)
Large Language Models (LLMs) like GPT-4 and Gemini have significantly advanced artificial intelligence by enabling machines to generate and comprehend human-like text. Despite their impressive capabilities, LLMs are not immune to limitations, including various biases. While much research has explored demographic biases, the cognitive biases in LLMs have not been equally scrutinized. This study delves into anchoring bias, a cognitive bias where initial information disproportionately influences judgment. Utilizing an experimental dataset, we examine how anchoring bias manifests in LLMs and verify the effectiveness of various mitigation strategies. Our findings highlight the sensitivity of LLM responses to biased hints. At the same time, our experiments show that, to mitigate anchoring bias, one needs to collect hints from comprehensive angles to prevent the LLMs from being anchored to individual pieces of information, while simple algorithms such as Chain-of-Thought, Thoughts of Principles, Ignoring Anchor Hints, and Reflection are not sufficient.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。