分析153篇CHI论文，揭示大模型如何改变人机交互研究
Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review
- 系统梳理2020-2024年153篇CHI论文中大模型的应用场景与角色
- 发现大模型在10个领域被应用，多数研究为实证与工具类贡献
- 指出研究普遍关注闭源模型，且存在可复现性与有效性问题
大语言模型（LLMs）被认为将重塑人机交互（HCI），不仅改变界面、设计模式与社会技术系统，也影响研究方法。然而，目前对大模型在HCI中的实际应用仍缺乏清晰认知。本文通过系统文献综述，分析了2020至2024年间153篇涉及大模型的CHI论文，构建了四个分类体系：（1）大模型的应用领域；（2）大模型在项目中的角色；（3）研究贡献类型；（4）已承认的局限与风险。研究发现，大模型应用于10个不同领域，主要以实证和工具类贡献为主。作者将大模型用于五种不同角色，包括研究工具或模拟用户。但多数研究仍聚焦于闭源模型，并频繁提及有效性和可复现性问题。文章提出改进人机交互研究的方法建议，并提供评估大模型相关工作的引导性问题。
原文摘要 · Abstract (English)
Large language models (LLMs) have been positioned to revolutionize HCI, by reshaping not only the interfaces, design patterns, and sociotechnical systems that we study, but also the research practices we use. To-date, however, there has been little understanding of LLMs' uptake in HCI. We address this gap via a systematic literature review of 153 CHI papers from 2020-24 that engage with LLMs. We taxonomize: (1) domains where LLMs are applied; (2) roles of LLMs in HCI projects; (3) contribution types; and (4) acknowledged limitations and risks. We find LLM work in 10 diverse domains, primarily via empirical and artifact contributions. Authors use LLMs in five distinct roles, including as research tools or simulated users. Still, authors often raise validity and reproducibility concerns, and overwhelmingly study closed models. We outline opportunities to improve HCI research with and on LLMs, and provide guiding questions for researchers to consider the validity and appropriateness of LLM-related work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。