用大模型辅助标注社交媒体疫苗争议中的道德框架，提升准确率与效率。
Can LLMs Assist Annotators in Identifying Morality Frames? -- Case Study on Vaccination Debate on Social Media
- 通过少量示例和解释让大模型生成道德框架概念
- 人类标注员在辅助下准确率提高，认知负担降低
- 适合需高效处理复杂心理语言任务的研究者
社交媒体在塑造公众舆论中起关键作用，尤其在疫苗等争议性议题上，不同道德视角影响个体立场。在自然语言处理中，由于数据稀缺及心理语言学任务的复杂性，仅靠人工标注成本高、耗时长，且易受认知负荷影响导致不一致。为此，我们利用大语言模型（LLMs）擅长通过少量上下文示例和解释实现快速适应新任务的能力，探索其在社交媒体疫苗争论中识别道德框架方面的辅助潜力。研究采用两步流程：先由LLMs生成概念与解释，再由人类标注员使用“思考发声”工具进行评估。结果表明，引入LLMs可显著提升标注准确率，降低任务难度与认知负荷，为复杂心理语言学任务中人机协作提供了可行路径。
原文摘要 · Abstract (English)
Nowadays, social media is pivotal in shaping public discourse, especially on polarizing issues like vaccination, where diverse moral perspectives influence individual opinions. In NLP, data scarcity and complexity of psycholinguistic tasks, such as identifying morality frames, make relying solely on human annotators costly, time-consuming, and prone to inconsistency due to cognitive load. To address these issues, we leverage large language models (LLMs), which are adept at adapting new tasks through few-shot learning, utilizing a handful of in-context examples coupled with explanations that connect examples to task principles. Our research explores LLMs' potential to assist human annotators in identifying morality frames within vaccination debates on social media. We employ a two-step process: generating concepts and explanations with LLMs, followed by human evaluation using a "think-aloud" tool. Our study shows that integrating LLMs into the annotation process enhances accuracy, reduces task difficulty, lowers cognitive load, suggesting a promising avenue for human-AI collaboration in complex psycholinguistic tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。