构建棋局推理对话数据集，研究多人协作解题机制。
DeliChess: A Multi-party Dialogue Dataset for Deliberation in Chess Puzzle Solving
- 设计多人协作解棋任务，记录独立回答与讨论后修正过程。
- 发现初始答案多样性越高，讨论后提升越大，部分组超越所有个体答案。
- 适合研究群体推理、对话动态及大模型辅助决策的有效性。
多参与者对话是研究协作推理与决策的关键场景，但现有数据集很少聚焦结构化、高推理强度的任务。我们提出DeliChess，一个多人协作解决选择题式国际象棋谜题的对话数据集。参与者先独立作答，再进行多方讨论并修改答案。数据集包含107段完整对话，附有讨论前后选项、话语级标注（沟通功能、认识立场、对讨论的贡献度）。分析显示，初始答案质量差异越大，讨论后提升越显著；答案演化轨迹揭示了讨论如何恢复、发现或丢失强方案。能超越所有个体答案的组别，往往表现出持续推理与开放认知态度。我们还设计了一个诊断性动作选择任务，发现测试的大模型与人类判断的有益行为仅有弱一致性。该数据集为建模群体推理、对话动态及有效讨论条件提供了实验平台。
原文摘要 · Abstract (English)
Multi-party dialogue is a critical setting for studying collaborative reasoning and decision-making, yet existing datasets rarely focus on structured, reasoning-intensive tasks. We introduce DeliChess, a dataset of group deliberation dialogues in which participants collaboratively solve multiple-choice chess puzzles. Participants first answer independently, then engage in multi-party deliberation and revise their individual answers. The dataset comprises 107 dialogues with full transcripts, pre- and post-deliberation choices, and utterance-level annotations of communicative function, epistemic stance, and usefulness for supporting deliberation. Our analyses show that greater diversity in initial solution quality is associated with larger gains, while answer trajectories reveal how deliberation can recover, discover, or lose strong answers. Cases in which groups surpass every independent answer are associated with sustained reasoning and epistemic openness. We further propose a diagnostic action-selection task and find that the tested LLMs show only weak agreement with human-attested helpful actions. Together, our dataset provides a testbed for modelling group reasoning, dialogue dynamics, and conditions for effective deliberation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。