对比五种大模型立场检测方法，发现提示词优于多智能体，且模型规模影响更大。
A Systematic Comparison of Prompting and Multi-Agent Methods for LLM-based Stance Detection
- 对比提示词与多智能体两种策略，评估14个子任务表现
- 最佳提示方法胜过最佳智能体方法，但后者需7-12倍更多API调用
- 模型规模影响大于方法选择，32B后性能趋于饱和
立场检测旨在识别文本作者对特定目标的态度。近期研究探索了从零样本提示到多智能体辩论等多种基于大模型的方法。然而，现有工作在数据划分、基础模型和评估协议上存在差异，难以进行公平比较。本文系统性地比较了五种方法——两类共五种策略：提示推理（直接提示、Auto-CoT、StSQA）和智能体辩论（COLA、MPRF），在四个数据集上的14个子任务中，使用来自六个模型家族的15个大模型（参数量从7B到72B+）。实验结果表明：第一，在所有完成测试的模型中，最佳提示方法优于最佳智能体方法，而后者每样本需7至12倍更多的API调用；第二，模型规模对性能的影响大于方法选择，性能在约32B时趋于饱和；第三，增强推理能力的模型（如DeepSeek-R1）并未在该任务上始终优于同规模的一般模型。
原文摘要 · Abstract (English)
Stance detection identifies the attitude of a text author toward a given target. Recent studies have explored various LLM-based strategies for this task, from zero-shot prompting to multi-agent debate. However, existing works differ in data splits, base models, and evaluation protocols, making fair comparison difficult. We conduct a systematic comparison that evaluates five methods across two categories -- prompt-based inference (Direct Prompting, Auto-CoT, StSQA) and agent-based debate (COLA, MPRF) -- on four datasets with 14 subtasks, using 15 LLMs from six model families with parameter sizes from 7B to 72B+. Our experiments yield several findings. First, on all models with complete results, the best prompt-based method outperforms the best agent-based method, while agent methods require 7 to 12 times more API calls per sample. Second, model scale has a larger impact on performance than method choice, with gains plateauing around 32B. Third, reasoning-enhanced models (DeepSeek-R1) do not consistently outperform general models of the same size on this task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。