研究发现大模型更像听话的工具,而非真正协作伙伴。
Human-AI collaboration or obedient and often clueless AI in instruct, serve, repeat dynamics?
- 用动态网络分析学生与AI的互动模式
- 发现指令主导、缺乏协同,且表现与难度无关
- 适合关注AI如何真正辅助人类思考的研究者
现有研究多聚焦语言学习,采用传统计数方法,忽视认知挑战任务中人机协作的演化过程。本研究通过定性编码与转移网络分析、序列分析、偏相关网络,结合卡方检验与个人残差阴影马赛克图,揭示学生与AI解决复杂问题时的互动模式及其随问题难度和表现的变化。结果表明,主导模式为‘指令型’,互动呈现反复下达指令而非协商合作,常出现学生提示与AI输出严重错位,体现协同不足。此外,任务复杂度、提示长度与学生成绩间无显著相关性,暗示缺乏认知深度或难度影响。研究指出当前大模型优化方向是遵从指令而非认知协作,限制其作为认知伙伴的能力。论文讨论了未来设计应优先考虑认知对齐与真正合作。
原文摘要 · Abstract (English)
While research on human-AI collaboration exists, it mainly examined language learning and used traditional counting methods with little attention to evolution and dynamics of collaboration on cognitively demanding tasks. This study examines human-AI interactions while solving a complex problem. Student-AI interactions were qualitatively coded and analyzed with transition network analysis, sequence analysis and partial correlation networks as well as comparison of frequencies using chi-square and Person-residual shaded Mosaic plots to map interaction patterns, their evolution, and their relationship to problem complexity and student performance. Findings reveal a dominant Instructive pattern with interactions characterized by iterative ordering rather than collaborative negotiation. Oftentimes, students engaged in long threads that showed misalignment between their prompts and AI output that exemplified a lack of synergy that challenges the prevailing assumptions about LLMs as collaborative partners. We also found no significant correlations between assignment complexity, prompt length, and student grades suggesting a lack of cognitive depth, or effect of problem difficulty. Our study indicates that the current LLMs, optimized for instruction-following rather than cognitive partnership, compound their capability to act as cognitively stimulating or aligned collaborators. Implications for designing AI systems that prioritize cognitive alignment and collaboration are discussed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。