大模型在无预设目标的立场检测中表现优于传统方法。
Can Large Language Models Address Open-Target Stance Detection?
- 提出开放目标立场检测新任务，不依赖预定义目标列表。
- 大模型在显性和隐性目标场景下均优于现有方法。
- 适合关注真实场景下文本立场分析的研究者。
立场检测（SD）旨在识别文本对某一目标的立场，通常分为支持、反对或无立场三类。本文提出开放目标立场检测（OTSD），这是最贴近实际的任务，其中目标既未在训练中出现，也未作为输入提供。我们评估了来自GPT、Gemini、Llama和Mistral系列的大语言模型（LLMs），并与唯一现有工作目标立场提取（TSE）进行对比，后者依赖预定义目标。相较于TSE，OTSD消除了对预设目标列表的依赖，使目标生成与评估更具挑战性。我们还提出一种目标质量评估指标，其与人类判断高度相关。实验表明，无论目标是否显式提及，大模型在目标生成上均优于TSE；同样，在立场检测方面，大模型整体表现也超越TSE。然而，当目标未明确提及时，大模型在目标生成和立场检测上仍存在明显困难。
原文摘要 · Abstract (English)
Stance detection (SD) identifies the text position towards a target, typically labeled as favor, against, or none. We introduce Open-Target Stance Detection (OTSD), the most realistic task where targets are neither seen during training nor provided as input. We evaluate Large Language Models (LLMs) from GPT, Gemini, Llama, and Mistral families, comparing their performance to the only existing work, Target-Stance Extraction (TSE), which benefits from predefined targets. Unlike TSE, OTSD removes the dependency of a predefined list, making target generation and evaluation more challenging. We also provide a metric for evaluating target quality that correlates well with human judgment. Our experiments reveal that LLMs outperform TSE in target generation, both when the real target is explicitly and not explicitly mentioned in the text. Similarly, LLMs overall surpass TSE in stance detection for both explicit and non-explicit cases. However, LLMs struggle in both target generation and stance detection when the target is not explicit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。