arXiv:2511.07641cs.CL2025-11被引 2

对比大模型与传统工具,发现小语种情感分析中老方法反而更准

LLMs vs. Traditional Sentiment Tools in Psychology: An Evaluation on Belgian-Dutch Narratives

  • 用三个荷兰语微调的大模型对比传统词典法
  • 2.5万条口语化文本中,词典法准确率高于大模型
  • 提示需为低资源语言定制评估体系

理解日常语言中的情感细微差别对计算语言学和情感研究至关重要。尽管传统基于词典的工具如LIWC和Pattern已作为基础工具使用,大语言模型(LLMs)则承诺更强的上下文理解能力。我们评估了三种荷兰语专用的LLMs(ChocoLlama-8B-Instruct、Reynaerde-7B-chat和GEITje-7B-ultra)在弗拉芒语(一种低资源语言变体)中的效价预测表现,与LIWC和Pattern进行对比。数据集包含约25000条来自102位荷语参与者自发提供的文本回应,每条回应附有自评效价评分(-50至+50)。出人意料的是,尽管架构先进,荷兰语微调的LLMs表现反而不如传统方法,其中Pattern表现最优。这些发现挑战了关于LLM在情感分析任务中天然优越性的假设,凸显了捕捉自发性真实叙事中情感效价的复杂性。结果强调了为低资源语言变体开发文化和语言定制的评估框架的必要性,同时质疑当前的LLM微调方法是否足以应对日常语言中细腻的情感表达。

原文摘要 · Abstract (English)

Understanding emotional nuances in everyday language is crucial for computational linguistics and emotion research. While traditional lexicon-based tools like LIWC and Pattern have served as foundational instruments, Large Language Models (LLMs) promise enhanced context understanding. We evaluated three Dutch-specific LLMs (ChocoLlama-8B-Instruct, Reynaerde-7B-chat, and GEITje-7B-ultra) against LIWC and Pattern for valence prediction in Flemish, a low-resource language variant. Our dataset comprised approximately 25000 spontaneous textual responses from 102 Dutch-speaking participants, each providing narratives about their current experiences with self-assessed valence ratings (-50 to +50). Surprisingly, despite architectural advancements, the Dutch-tuned LLMs underperformed compared to traditional methods, with Pattern showing superior performance. These findings challenge assumptions about LLM superiority in sentiment analysis tasks and highlight the complexity of capturing emotional valence in spontaneous, real-world narratives. Our results underscore the need for developing culturally and linguistically tailored evaluation frameworks for low-resource language variants, while questioning whether current LLM fine-tuning approaches adequately address the nuanced emotional expressions found in everyday language use.

情感分析大模型低资源语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。