arXiv:2508.01390cs.CYcs.AI2025-08被引 6

大模型污染正威胁线上行为研究,导致数据失真与结论不可靠。

Recognising, Anticipating, and Mitigating LLM Pollution of Online Behavioural Research

  • 识别出三种污染形式:部分使用、完全代理和行为预判。
  • 污染会扭曲样本真实性,引入难以察觉的系统性偏差。
  • 适合关注线上研究方法论安全的研究者与平台方阅读。

在线行为研究正面临新兴威胁:参与者越来越多地使用大语言模型(LLMs)获取建议、翻译或任务代劳,即“大模型污染”。我们识别出三种相互作用的污染形式:第一,部分中介指参与者选择性使用LLM处理任务特定环节(如翻译或措辞),导致研究人员误将模型生成内容当作人类行为;第二,完全代理指自主型LLM在极少人类监督下完成研究,从根本上动摇以人为对象研究的核心前提;第三,大模型溢出指参与者即使在无模型参与的研究中,也因预期存在大模型而改变自身行为。这三类污染形成连续谱系,并引发级联式偏差,破坏样本真实性,引入难检测的偏见,最终削弱线上研究对人类认知与行为的学术基础。更关键的是,该威胁正随生成式AI发展同步演进,形成方法论军备竞赛。为此,我们提出涵盖研究实践、平台责任与社区协作的多层次应对策略。唯有协同适应,方能保障线上行为研究的方法论完整性与有效性。

原文摘要 · Abstract (English)

Online behavioural research faces an emerging threat as participants increasingly turn to large language models (LLMs) for advice, translation, or task delegation: LLM Pollution. We identify three interacting variants through which LLM Pollution threatens the validity and integrity of online behavioural research. First, Partial LLM Mediation occurs when participants make selective use of LLMs for specific aspects of a task, such as translation or wording support, leading researchers to (mis)interpret LLM-shaped outputs as human ones. Second, Full LLM Delegation arises when agentic LLMs complete studies with little to no human oversight, undermining the central premise of human-subject research at a more foundational level. Third, LLM Spillover signifies human participants altering their behaviour as they begin to anticipate LLM presence in online studies, even when none are involved. While Partial Mediation and Full Delegation form a continuum of increasing automation, LLM Spillover reflects second-order reactivity effects. Together, these variants interact and generate cascading distortions that compromise sample authenticity, introduce biases that are difficult to detect post hoc, and ultimately undermine the epistemic grounding of online research on human cognition and behaviour. Crucially, the threat of LLM Pollution is already co-evolving with advances in generative AI, creating an escalating methodological arms race. To address this, we propose a multi-layered response spanning researcher practices, platform accountability, and community efforts. As the challenge evolves, coordinated adaptation will be essential to safeguard methodological integrity and preserve the validity of online behavioural research.

大模型污染线上研究方法论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。