用用户历史发言总结提升大模型在政治立场判断中的准确率
Exploiting contextual information to improve stance detection in informal political discourse with LLMs
- 用用户历史发言生成意识形态摘要作为上下文提示
- 准确率最高提升38.5%，达74%超过此前方法
- 精选政治内容比大量随机内容更有效
本研究探讨大型语言模型（LLMs）在非正式政治网络话语中的立场检测效果，此类语境中语言常具讽刺、模糊和依赖上下文的特点。我们考察提供用户画像摘要（基于历史发帖生成）是否能提升分类准确率。利用真实政治论坛数据集，构建包含意识形态倾向、高频话题和语言模式的结构化用户画像。在七种先进LLM上进行基准与增强上下文设置的综合对比评估。结果显示，引入上下文提示显著提升准确率，提升幅度达17.5%至38.5%，最高达74%，超越以往方法。分析还表明，合理选择政治相关内容比单纯扩大上下文规模更有效。研究证明，融入用户层面上下文可显著提升大模型在复杂政治分类任务中的表现。
原文摘要 · Abstract (English)
This study investigates the use of Large Language Models (LLMs) for political stance detection in informal online discourse, where language is often sarcastic, ambiguous, and context-dependent. We explore whether providing contextual information, specifically user profile summaries derived from historical posts, can improve classification accuracy. Using a real-world political forum dataset, we generate structured profiles that summarize users' ideological leaning, recurring topics, and linguistic patterns. We evaluate seven state-of-the-art LLMs across baseline and context-enriched setups through a comprehensive cross-model evaluation. Our findings show that contextual prompts significantly boost accuracy, with improvements ranging from +17.5\% to +38.5\%, achieving up to 74\% accuracy that surpasses previous approaches. We also analyze how profile size and post selection strategies affect performance, showing that strategically chosen political content yields better results than larger, randomly selected contexts. These findings underscore the value of incorporating user-level context to enhance LLM performance in nuanced political classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。