让大模型从海量对话中提炼集体观点，发现特定人群的新兴关切。
From Chat Logs to Collective Insights: Aggregative Question Answering
- 设计新任务：从数千条对话中聚合信息回答群体性问题。
- 构建数据集WildChat-AQA，含6027个真实对话衍生的问题。
- 揭示现有方法在推理效率与准确性上均存短板，需新算法支持。
由大语言模型驱动的对话智能体正迅速融入日常交互，产生前所未有的对话数据量。这些数据为洞察社会兴趣、流行话题及集体关注提供了强大视角。然而，现有方法通常将互动视为独立事件，忽略了大规模对话日志中蕴含的聚合推理价值。本文提出「聚合式问答」新任务，要求模型显式地对成千上万条用户-聊天机器人对话进行推理，以回答如‘特定群体中浮现的新关切’等聚合性问题。为此，我们构建了基准数据集WildChat-AQA，包含6,027个源自182,330条真实对话的聚合问题。实验表明，现有方法或难以有效推理，或计算成本过高，凸显了开发新方法以从大规模对话数据中提取集体洞见的必要性。
原文摘要 · Abstract (English)
Conversational agents powered by large language models (LLMs) are rapidly becoming integral to our daily interactions, generating unprecedented amounts of conversational data. Such datasets offer a powerful lens into societal interests, trending topics, and collective concerns. Yet, existing approaches typically treat these interactions as independent and miss critical insights that could emerge from aggregating and reasoning across large-scale conversation logs. In this paper, we introduce Aggregative Question Answering, a novel task requiring models to reason explicitly over thousands of user-chatbot interactions to answer aggregative queries, such as identifying emerging concerns among specific demographics. To enable research in this direction, we construct a benchmark, WildChat-AQA, comprising 6,027 aggregative questions derived from 182,330 real-world chatbot conversations. Experiments show that existing methods either struggle to reason effectively or incur prohibitive computational costs, underscoring the need for new approaches capable of extracting collective insights from large-scale conversational data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。