发现大模型倾向于答‘不’，与人类顺从倾向相反。
Acquiescence Bias in Large Language Models
- 用多语言任务测试不同模型的应答倾向
- 结果显示模型普遍偏好回答‘否’
- 揭示模型响应偏差，适合研究可信度的学者
顺从偏差指人类在问卷中倾向于同意陈述，无论真实态度如何，已有充分研究。由于大语言模型（LLMs）对输入微小变化敏感，且训练数据来自人类生成内容，推测其可能也存在类似倾向。我们研究了不同模型、任务和语言（英语、德语、波兰语）下LLMs是否存在顺从偏差。结果表明，与人类相反，LLMs表现出对‘否’的系统性偏好，无论该回答是表示同意还是不同意。这一发现揭示了模型在响应中潜在的非对称偏差，提示需警惕其在决策支持中的可靠性。
原文摘要 · Abstract (English)
Acquiescence bias, i.e. the tendency of humans to agree with statements in surveys, independent of their actual beliefs, is well researched and documented. Since Large Language Models (LLMs) have been shown to be very influenceable by relatively small changes in input and are trained on human-generated data, it is reasonable to assume that they could show a similar tendency. We present a study investigating the presence of acquiescence bias in LLMs across different models, tasks, and languages (English, German, and Polish). Our results indicate that, contrary to humans, LLMs display a bias towards answering no, regardless of whether it indicates agreement or disagreement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。