用AI自适应地进行深度访谈,既覆盖预设话题又捕捉突发灵感。
SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery
- 通过模拟对话推演,动态选择高价值问题以平衡主题覆盖与探索深度。
- 相比基线提升4.7%主题覆盖率,减少对话轮次并激发更丰富的新兴洞察。
- 适合需要高质量定性数据的研究者或产品团队,尤其关注人机交互设计。
用户经验中的定性洞察对产品和政策决策至关重要,但大规模收集受限于专家访谈时间与人力。现有基于大语言模型(LLM)的自动访谈系统缺乏在预设主题覆盖与动态追问、深度挖掘及自然浮现主题之间的平衡机制。本文将自适应半结构化访谈建模为优化问题,定义访谈效用为预设话题覆盖、新兴主题发现与访谈成本(对话轮次)之间的权衡。基于此,提出SparkMe——一个多代理LLM访谈系统,通过模拟对话推演进行审慎规划,选择预期效用最高的问题。在基于LLM的模拟受访者上的控制实验表明,SparkMe显著提升访谈效用:主题指南覆盖率提升4.7%(优于最佳基线),同时生成更丰富的新兴洞察,且对话轮次更少。进一步在70名来自7个职业领域的参与者中开展用户研究,验证其在人工智能对工作流影响议题上的有效性。领域专家评价认为,SparkMe生成的访谈具有高度适应性,能揭示先前方法未捕捉到的职业特定见解。代码、数据集与评估协议已开源:https://github.com/SALT-NLP/SparkMe。
原文摘要 · Abstract (English)
Qualitative insights from user experiences are critical for informing product and policy decisions, but collecting such data at scale is constrained by the time and availability of experts to conduct semi-structured interviews. Recent work has explored using large language models (LLMs) to automate interviewing, yet existing systems lack a principled mechanism for balancing systematic coverage of predefined topics with adaptive exploration, or the ability to pursue follow-ups, deep dives, and emergent themes that arise organically during conversation. In this work, we formulate adaptive semi-structured interviewing as an optimization problem over the interviewer's behavior. We define interview utility as a trade-off between coverage of a predefined interview topic guide, discovery of relevant emergent themes, and interview cost measured by length. Based on this formulation, we introduce SparkMe, a multi-agent LLM interviewer that performs deliberative planning via simulated conversation rollouts to select questions with high expected utility. We evaluate SparkMe through controlled experiments with LLM-based interviewees, showing that it achieves higher interview utility, improving topic guide coverage (+4.7% over the best baseline) and eliciting richer emergent insights while using fewer conversational turns than prior LLM interviewing approaches. We further validate SparkMe in a user study with 70 participants across 7 professions on the impact of AI on their workflows. Domain experts rate SparkMe as producing high-quality adaptive interviews that surface helpful profession-specific insights not captured by prior approaches. The code, datasets, and evaluation protocols for SparkMe are available as open-source at https://github.com/SALT-NLP/SparkMe.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。