用小模型验证大模型判断,让分析更准又省资源。
Collaborative Stance Detection via Small-Large Language Model Consistency Verification
- 大模型批量处理+小模型逻辑校验,协同判断立场
- 零样本下超越现有方法,每条推文仅需0.54次大模型调用
- 适合需要低成本、高效率的社会媒体监控场景
社交媒体立场检测旨在识别推文中对特定目标表达的态度。当前研究普遍倾向使用大语言模型(LLM),因其性能显著提升。然而,过度依赖昂贵的LLM进行大规模社交媒体分析在实际系统中不切实际。为此,我们提出协同立场检测框架CoVer,通过小-大模型一致性验证提升LLM利用率。CoVer采用共享上下文的批量推理方式,由LLM生成立场判断及解释;随后引入小语言模型(SLM)进行逻辑一致性验证,剔除上下文噪声干扰;最后对多次出现低一致性的文本,采用加权聚合历史LLM预测结果进行分类。实验表明,CoVer在多个基准上实现零样本最优表现,每条推文仅需0.54次LLM调用,显著提升性能与实用性。
原文摘要 · Abstract (English)
Stance detection on social media aims to identify attitudes expressed in tweets towards specific targets. Current studies prioritize Large Language Models (LLMs) over Small Language Models (SLMs) due to the overwhelming performance improving provided by LLMs. However, heavily relying on LLMs for stance detection, regardless of the cost, is impractical for real-world social media monitoring systems that require vast data analysis. To this end, we propose \textbf{\underline{Co}}llaborative Stance Detection via Small-Large Language Model Consistency \textbf{\underline{Ver}}ification (\textbf{CoVer}) framework, which enhances LLM utilization via context-shared batch reasoning and logical verification between LLM and SLM. Specifically, instead of processing each text individually, CoVer processes texts batch-by-batch, obtaining stance predictions and corresponding explanations via LLM reasoning in a shared context. Then, to exclude the bias caused by context noises, CoVer introduces the SLM for logical consistency verification. Finally, texts that repeatedly exhibit low logical consistency are classified using consistency-weighted aggregation of prior LLM stance predictions. Our experiments show that CoVer outperforms state-of-the-art methods across multiple benchmarks in the zero-shot setting, achieving 0.54 LLM queries per tweet while significantly enhancing performance. Our CoVer offers a more practical solution for LLM deploying for social media stance detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。