用AI自动化分析人类思考过程，让大规模认知研究成为可能。
Scaling up the think-aloud method
- 用NLP自动转录和编码思考过程，构建搜索图谱
- 640人参与，发现人类解题路径具中等一致性
- 为大规模认知研究提供可扩展的分析工具
思考出声法通过让用户在解题时实时表达想法，能获取丰富的人类推理数据。然而，由于转录与标注耗时，该方法在当代认知科学中逐渐式微。本文开发了基于自然语言处理的自动化转录与标注方法，实现了对思考数据的大规模分析。在研究中，640名参与者在玩数学推理游戏“24点”时进行思考出声，系统自动转录录音并将其编码为搜索图谱，与人工标注达到中等一致性。我们分析这些图谱，刻画了人类推理轨迹的一致性与差异性。本工作证明了大规模思考出声数据的价值，也为口头报告的自动化分析提供了可行性验证。
原文摘要 · Abstract (English)
The think-aloud method, where participants voice their thoughts as they solve a task, is a valuable source of rich data about human reasoning processes. Yet, it has declined in popularity in contemporary cognitive science, largely because labor-intensive transcription and annotation preclude large sample sizes. Here, we develop methods to automate the transcription and annotation of verbal reports of reasoning using natural language processing tools, allowing for large-scale analysis of think-aloud data. In our study, 640 participants thought aloud while playing the Game of 24, a mathematical reasoning task. We automatically transcribed the recordings and coded the transcripts as search graphs, finding moderate inter-rater reliability with humans. We analyze these graphs and characterize consistency and variation in human reasoning traces. Our work demonstrates the value of think-aloud data at scale and serves as a proof of concept for the automated analysis of verbal reports.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。