剖析人机交互领域'预实验'的模糊现状与报告问题
What Do We Mean by 'Pilot Study': Early Findings from a Meta-Review of Pilot Study Reporting at CHI
- 通过元综述分析CHI论文中预实验的使用情况
- 发现多数预实验缺乏设计细节和对主研究的影响说明
- 呼吁建立统一定义与报告规范，提升研究可复现性
预实验在人机交互（HCI）研究中极为普遍，许多CHI论文引用‘预实验’、‘预测试’或‘初步研究’来支持设计决策、验证流程或解释方法选择。然而，尽管频率高，预实验在HCI中的角色仍概念模糊且缺乏实证研究。与医学、护理和教育等领域相比，这些领域已有明确的预实验定义、指南、报告标准甚至专门期刊，而CHI社区尚未形成对预实验内涵、目的及报告方式的共识。大量论文仅泛泛提及预实验，未提供设计、结果或其如何影响主研究的具体信息。这种不一致性表明我们领域存在方法论盲区。
原文摘要 · Abstract (English)
Pilot studies (PS) are ubiquitous in HCI research. CHI papers routinely reference 'pilot studies', 'pilot tests', or 'preliminary studies' to justify design decisions, verify procedures, or motivate methodological choices. Yet despite their frequency, the role of pilot studies in HCI remains conceptually vague and empirically underexamined. Unlike fields such as medicine, nursing, and education, where pilot and feasibility studies have well-established definitions, guidelines, reporting standards and even a dedicated research journal, the CHI community lacks a shared understanding of what constitutes a pilot study, why they are conducted, and how they should be reported. Many papers reference pilots 'in passing', without details about design, outcomes, or how the pilot informed the main study. This variability suggests a methodological blind spot in our community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。