arXiv:2411.19007cs.CL2024-11

分析维基百科讨论页中同一用户自答现象,揭示其背后多重动机。

Talking to oneself in CMC: a study of self replies in Wikipedia talk pages

  • 通过词法分析识别自答内容特征,构建七类行为分类体系。
  • 在英法双语各100条数据上验证分类有效性,人类标注者表现稳定。
  • 大模型在多类别识别上存在困难,凸显自答行为的复杂性。

本研究对维基百科讨论页中由同一作者发出的连续两条消息进行定性分析,这类自答模式在两消息及以上讨论中占比超过10%。通过对第二条消息的词汇特征初步考察,提出一个包含七类行为的分类体系,并用于标注英语和法语各100条参考样本。最终分析显示,人类标注者整体一致性良好,而经过指令微调的大语言模型在多个类别上表现出显著识别困难,表明此类自答行为具有复杂的语用与认知动因。

原文摘要 · Abstract (English)

This study proposes a qualitative analysis of self replies in Wikipedia talk pages, more precisely when the first two messages of a discussion are written by the same user. This specific pattern occurs in more than 10% of threads with two messages or more and can be explained by a number of reasons. After a first examination of the lexical specificities of second messages, we propose a seven categories typology and use it to annotate two reference samples (English and French) of 100 threads each. Finally, we analyse and compare the performance of human annotators (who reach a reasonable global efficiency) and instruction-tuned LLMs (which encounter important difficulties with several categories).

人机交互文本分析维基百科对话行为

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。