分析20万条科研对话数据，揭示用户如何与AI科研助手协作。
Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset
- 收集两工具20万+交互日志，构建真实科研场景数据集
- 用户提问更长更复杂，把AI当研究伙伴并反复迭代输出
- 适合设计下一代智能科研助手，支持真实评估
AI驱动的科研工具正快速融入研究流程，但对其真实使用情况的理解仍不足。本文提出并分析了Asta Interaction Dataset，一个大规模数据集，包含来自两个部署工具（文献发现界面与科学问答界面）超过20万条用户查询和交互日志，这些工具运行于基于大模型的检索增强生成平台。通过该数据集，我们刻画了查询模式、参与行为及使用随经验的变化。发现用户提交的查询比传统搜索更长更复杂，并将系统视为协作研究伙伴，委托其完成内容起草与研究空白识别等任务。用户将生成结果视为持续存在的工作成果，以非线性方式反复回溯和跳转输出与引用证据。随着经验积累，用户提出更精准的问题，更深入地使用支撑引文，但关键词式提问仍普遍存在。我们公开匿名化数据集与新提出的查询意图分类体系，以指导未来真实场景中AI科研助手的设计，并支持更真实的评估。
原文摘要 · Abstract (English)
AI-powered scientific research tools are rapidly being integrated into research workflows, yet the field lacks a clear lens into how researchers use these systems in real-world settings. We present and analyze the Asta Interaction Dataset, a large-scale resource comprising over 200,000 user queries and interaction logs from two deployed tools (a literature discovery interface and a scientific question-answering interface) within an LLM-powered retrieval-augmented generation platform. Using this dataset, we characterize query patterns, engagement behaviors, and how usage evolves with experience. We find that users submit longer and more complex queries than in traditional search, and treat the system as a collaborative research partner, delegating tasks such as drafting content and identifying research gaps. Users treat generated responses as persistent artifacts, revisiting and navigating among outputs and cited evidence in non-linear ways. With experience, users issue more targeted queries and engage more deeply with supporting citations, although keyword-style queries persist even among experienced users. We release the anonymized dataset and analysis with a new query intent taxonomy to inform future designs of real-world AI research assistants and to support realistic evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。