对比人类与大模型在信息查询中的回答策略差异。
DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering
- 用话语结构理论分析回答中的修辞行为
- 人类社区有不同构建答案的偏好，大模型则缺乏多样性
- 大模型更倾向广泛覆盖，常回应人类忽略的解释
我们提出DiscoTrace，一种识别回答者在信息查询类问题中使用修辞策略的方法。该方法将答案表示为一系列与问题相关的语篇行为，并附上对原问题的解释，基于修辞结构理论解析生成。对九个不同社区的回答应用DiscoTrace发现，各社区在答案构建上有显著偏好差异；相比之下，即使被提示模仿特定人类社区的答题规范，大模型仍不表现出修辞多样性，且系统性地选择更广的覆盖面，涵盖人类回答者有意忽略的问题解释。DiscoTrace揭示的丰富、社区敏感的回答行为结构，可指导更具语用意识的大模型回答系统开发。
原文摘要 · Abstract (English)
We introduce DiscoTrace, a method to identify the rhetorical strategies answerers use when responding to information-seeking questions. DiscoTrace represents answers as a sequence of question-related discourse acts paired with interpretations of the original question, annotated on top of rhetorical structure theory parses. Applying DiscoTrace to answers from nine different communities reveals that communities have diverse preferences for answer construction. In contrast, LLMs do not exhibit rhetorical diversity in their answers, even when prompted to mimic specific human community answering guidelines. LLMs also systematically opt for breadth, addressing interpretations of questions that human answerers choose not to address. The rich, community-sensitive answering behavior structurally revealed by DiscoTrace can guide the development of pragmatic LLM answerers that are more attuned to contextual information needs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。