用大模型推理过程指导小模型学习,提升低资源语言立场检测效果
Rationale-Guided Knowledge Distillation for Cross-Lingual Stance Detection

- 用思维链提示让大模型生成推理过程,提炼为小模型知识
- 在多语言数据集上优于现有方法,显著提升低资源语言表现
- 适合需要高效跨语言立场分析的应用场景
立场检测旨在识别文本对特定目标的态度倾向,是众多下游应用的重要任务。尽管现有研究在英语等资源丰富语言中表现良好,但许多低资源语言(如加泰罗尼亚语)仍缺乏足够标注数据。跨语言立场检测通过将高资源语言的知识迁移到低资源语言来缓解此问题。然而,现有方法主要依赖文本与目标的语义对齐,忽视了可靠立场判断所需的推理过程。虽然大语言模型具备强大推理能力,但其高昂的计算成本和延迟限制了实际部署。为此,我们提出一种基于推理过程引导的知识蒸馏框架。具体而言,利用思维链提示引导大语言模型生成有意义的推理过程,并将所得推理知识蒸馏到轻量级学生模型中。我们还设计了双路径蒸馏机制,对增强推理与无推理表示及其预测分布进行对齐。此外,引入两种对比学习策略以增强立场区分能力。在多语言基准测试中,我们的方法持续优于竞争基线。
原文摘要 · Abstract (English)
Stance detection aims to identify whether a text expresses a favorable or opposing attitude toward a given target, and serves as an important task for various downstream applications. Although existing studies have achieved strong performance in monolingual settings, especially in English, many low-resource languages such as Catalan still lack sufficient annotated data for training effective models. Cross-lingual stance detection alleviates this problem by transferring stance knowledge from resource-rich languages to low-resource languages. However, most existing methods mainly rely on semantic alignment between texts and targets, while ignoring the reasoning process required for reliable stance inference. Although Large Language Models provide strong reasoning ability, their high computational cost and inference latency limit practical deployment. To address these limitations, we propose a rationale-guided knowledge distillation framework for cross-lingual stance detection. Specifically, we use Chain-of-Thought prompting to guide Large Language Models in generating informative rationales, and distill the resulting reasoning knowledge into a compact student model. We further design a dual-path distillation mechanism to align rationale-enhanced and rationale-free representations, together with their prediction distributions. In addition, two contrastive learning strategies are introduced to improve stance discrimination. Experiments on multilingual benchmarks demonstrate that our method consistently outperforms competitive baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。