arXiv:2506.04139cs.CL2025-06

测试大模型在弗莱芒语情感分析中的表现,发现仍不如传统词典工具。

Are Lexicon-Based Tools Still the Gold Standard for Valence Analysis in Low-Resource Flemish?

  • 用2.5万条弗莱芒语日常叙述数据,测试三个荷兰语大模型的情感判断能力。
  • 大模型预测情感值的准确率低于经典词典工具LIWC和Pattern。
  • 强调需为低资源语言构建本土化数据集与微调模型,提升真实场景分析能力。

理解日常语言的细微差别对计算语言学与情绪研究至关重要。传统词典工具如LIWC和Pattern长期作为该领域基础工具:LIWC是社会科学中验证最充分的词频分析工具,Pattern则是提供NLP功能的开源Python库。然而日常语言具有自发性、表达丰富且高度依赖语境。为探究大模型在捕捉弗莱芒语日常叙事情感值方面的能力,我们收集了102位荷语使用者约2.5万条文本回应,每条回答均附有-50至+50连续量表的自评情感值。随后评估了三个专为荷兰语优化的大模型在预测这些情感值上的表现,并与LIWC和Pattern的结果对比。结果表明,尽管大模型架构不断进步,但在处理自发性、真实语境下的语言时,其情感值捕捉能力仍不及传统词典工具。本研究强调必须开发文化与语言适配的模型,以应对自然语言使用中的复杂性。提升自动化情感分析不仅推动计算方法发展,也助力心理学研究获得生态有效的人类日常体验洞察。我们呼吁加强为弗莱芒语等低资源语言构建全面数据集并进行模型微调,弥合计算语言学与情绪研究之间的差距。

原文摘要 · Abstract (English)

Understanding the nuances in everyday language is pivotal for advancements in computational linguistics & emotions research. Traditional lexicon-based tools such as LIWC and Pattern have long served as foundational instruments in this domain. LIWC is the most extensively validated word count based text analysis tool in the social sciences and Pattern is an open source Python library offering functionalities for NLP. However, everyday language is inherently spontaneous, richly expressive, & deeply context dependent. To explore the capabilities of LLMs in capturing the valences of daily narratives in Flemish, we first conducted a study involving approximately 25,000 textual responses from 102 Dutch-speaking participants. Each participant provided narratives prompted by the question, "What is happening right now and how do you feel about it?", accompanied by self-assessed valence ratings on a continuous scale from -50 to +50. We then assessed the performance of three Dutch-specific LLMs in predicting these valence scores, and compared their outputs to those generated by LIWC and Pattern. Our findings indicate that, despite advancements in LLM architectures, these Dutch tuned models currently fall short in accurately capturing the emotional valence present in spontaneous, real-world narratives. This study underscores the imperative for developing culturally and linguistically tailored models/tools that can adeptly handle the complexities of natural language use. Enhancing automated valence analysis is not only pivotal for advancing computational methodologies but also holds significant promise for psychological research with ecologically valid insights into human daily experiences. We advocate for increased efforts in creating comprehensive datasets & finetuning LLMs for low-resource languages like Flemish, aiming to bridge the gap between computational linguistics & emotion research.

情感分析大模型低资源语言弗莱芒语

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。