大模型在信息不对称下合作沟通能力差,常违背对话准则却不知。
Flout at Your Own Risk: LLMs Struggle with Pragmatic Cooperativity Under Epistemic Asymmetry

- 设计多角色协作任务,测试模型在信息不全时的对话协作能力。
- 发现大模型在信息缺失时仍会违反合作原则,导致沟通失败。
- 适合研究对话智能、人机协作或大模型安全性的学者参考。
高效协作依赖于合作性沟通,包括利用上下文线索进行推理。随着大模型在协作与代理流程中的广泛应用,其是否具备此类语用能力成为关键问题,尤其在与合作者信息不对称的场景中。本文首次在部分信息条件下,对大模型在多方协作任务中的语用推理能力进行系统研究。我们形式化了协作认知不对称的概念,将客观任务成功与格赖斯的合作原则相联系,并实证评估了多种大模型在作为说话者和听者时的合作能力,涵盖提示工程与后训练策略。结果表明,尽管大模型在协作中表现出一定语用能力,且可通过提示或后训练激发,但在信息不完整时仍面临沟通挑战,某些失效模式与未被察觉地违背格赖斯准则密切相关。
原文摘要 · Abstract (English)
Fruitful collaborations rely on cooperative communications, including of contextual cues to incorporate into reasoning. The increasing use of LLMs in collaborative and agentic pipelines raises questions about the extent to which they exhibit these pragmatic capabilities, especially in scenarios where they may not have access to the same information as their collaborators. In this paper, we perform a novel investigation into the pragmatic reasoning capabilities of LLMs in a multi-party collaborative task under partial information conditions. We formalize a notion of collaborative epistemic asymmetry that explicitly connects objective task success to Grice's cooperative principle and empirically assess various LLMs' abilities to act cooperatively as both speakers and listeners, including both prompting and post-training strategies. Our results show that while LLMs exhibit certain pragmatic capabilities in collaborative settings, and these can be elicited through prompting and post-training, they still face challenges in pragmatic communication with incomplete information, and that certain failure modes do correlate with floutings of Grice's maxims that go unrecognized.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。