用跨语言功能向量实现零样本情绪识别,无需示例也能跨语言迁移。
Cross-lingual Functional Vectors for Emotion Detection in Large Language Models

- 从源语言提取功能向量,无须示例即可引导目标语言任务
- 在多语言情绪识别上显著提升性能,跨语言效果稳定
- 只需激活特定注意力头,轻量高效适合大规模应用
功能向量(FVs)作为一种新兴机制,可通过注入基于上下文示例提取的任务特定潜在方向,调控大语言模型(LLM)行为。尽管已有研究证明其在结构化上下文学习中的有效性,但其在语义复杂任务中的表现以及跨语言泛化能力仍不明确。本文以多语言多标签情绪识别为挑战性基准,探究FVs的跨语言可迁移性。具体而言,考察是否可在标准零样本和扰动零样本设置下,仅凭源语言提取的FVs,在推理时不提供示例的情况下引导目标语言的任务行为。实验表明,跨多种语言设置,引入FVs均能显著提升性能,说明其捕捉的是语言无关的任务相关信号而非纯粹的语言特异性词汇模式,凸显其作为轻量级、可迁移的多语言任务适配机制的潜力。我们还发现,每种LLM在构建有效FVs时存在相对稳定的最优注意力头区间,且该模式在不同语言间保持一致。此外,FVs可部分复现标准少样本上下文学习的任务引导效果,同时避免处理多个示例带来的计算开销,适用于大规模实际场景。代码已开源。
原文摘要 · Abstract (English)
Function vectors (FVs) have recently emerged as a promising mechanism for steering the behavior of large language models (LLMs) by injecting task-specific latent direction representations derived from in-context demonstrations. While prior studies have shown that FVs can recover task behavior in structured in-context learning settings, their effectiveness on semantically complex tasks and their ability to generalize across languages remain underexplored. We investigate the cross-lingual transferability of FVs using multilingual multi-label emotion recognition as a challenging semantic classification benchmark. Specifically, we examine whether FVs extracted from a source language can steer task behavior in another language under both standard clean and perturbed zero-shot settings without providing demonstrations during inference. Across diverse cross-lingual settings, applying FVs substantially improves performance, suggesting that FVs capture language-agnostic, task-relevant signals rather than purely language-specific lexical patterns, and highlighting their potential as a lightweight and transferable mechanism for multilingual task adaptation. We observe that each LLM exhibits a relatively stable optimal range of attention heads for constructing effective FVs, and the pattern remains consistent across languages. In addition, FVs can partially replicate the task-steering effects of standard few-shot in-context learning while avoiding the computational overhead of processing multiple demonstrations, making them effective for large-scale practical applications. Our code is available at https://github.com/yingjie7/cross_lingual_fvs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。