arXiv:2409.16807cs.CL2024-09被引 3

定义网络气候争论中伪善指控的独立任务并发现识别难点

A Few Hypocrites: Few-Shot Learning and Subtype Definitions for Detecting Hypocrisy Accusations in Online Climate Change Debates

  • 将伪善指控作为独立NLP任务,区分个人与政治两类
  • 6样本少样本学习下模型F1达0.68,优于此前0.44
  • 模型在识别政治伪善上显著弱于个人道德伪善

气候变化是网络讨论中的关键议题,伪善指控是其中的核心修辞手段。然而,在大规模文本分析中,伪善指控检测仍属研究不足的任务,通常被视为谬误论证检测的子任务。本文首次将伪善指控检测定义为NLP中的独立任务,并识别出相关子类型。我们构建了气候伪善指控语料库(CHAC),包含420条Reddit气候辩论评论,由专家标注为两类:个人伪善与政治伪善。评估了6样本上下文学习与3个指令微调的大语言模型在该数据集上的表现。结果显示,GPT-4o和Llama-3模型在检测上展现出潜力(F1达0.68),而此前工作仅为0.44。但针对复杂语义概念如伪善,上下文至关重要,模型尤其难以识别政治伪善,相较个人道德伪善表现更差。本研究为伪善检测与气候话语分析提供新洞见,是大规模在线气候争论中伪善指控分析的重要起点。

原文摘要 · Abstract (English)

The climate crisis is a salient issue in online discussions, and hypocrisy accusations are a central rhetorical element in these debates. However, for large-scale text analysis, hypocrisy accusation detection is an understudied tool, most often defined as a smaller subtask of fallacious argument detection. In this paper, we define hypocrisy accusation detection as an independent task in NLP, and identify different relevant subtypes of hypocrisy accusations. Our Climate Hypocrisy Accusation Corpus (CHAC) consists of 420 Reddit climate debate comments, expert-annotated into two different types of hypocrisy accusations: personal versus political hypocrisy. We evaluate few-shot in-context learning with 6 shots and 3 instruction-tuned Large Language Models (LLMs) for detecting hypocrisy accusations in this dataset. Results indicate that the GPT-4o and Llama-3 models in particular show promise in detecting hypocrisy accusations (F1 reaching 0.68, while previous work shows F1 of 0.44). However, context matters for a complex semantic concept such as hypocrisy accusations, and we find models struggle especially at identifying political hypocrisy accusations compared to personal moral hypocrisy. Our study contributes new insights in hypocrisy detection and climate change discourse, and is a stepping stone for large-scale analysis of hypocrisy accusation in online climate debates.

伪善检测少样本学习气候话语

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。