让语言模型具备动态文化自知能力,提升跨文化任务表现。
CALM: Culturally Self-Aware Language Models
- 分离显性文化概念与隐含文化信号,通过对比学习构建结构化文化簇。
- 用专家混合机制按文化维度自适应融合,实现精细交互。
- 通过自我反思学习持续优化文化认知,适合跨文化应用研究者。
语言模型的文化意识是指理解并适应不同文化背景的能力。然而,现有方法大多将文化视为静态背景知识,忽略了其动态演化特性,限制了在需要真实文化敏感性的下游任务中的可靠性。本文提出CALM框架,赋予语言模型文化自知能力。CALM通过对比学习,将任务语义与显性文化概念及隐含文化信号解耦,并形成结构化的文化簇;再通过交叉注意力对齐相关文化特征,建立细粒度交互;最后利用专家混合机制沿文化特定维度自适应融合。由此生成的统一表征与模型原始知识融合,构建出基于文化的内部身份状态,并通过自我提示的反思学习不断优化,实现持续适应与自我修正。在多个跨文化基准数据集上的大量实验表明,CALM始终优于当前最优方法。
原文摘要 · Abstract (English)
Cultural awareness in language models is the capacity to understand and adapt to diverse cultural contexts. However, most existing approaches treat culture as static background knowledge, overlooking its dynamic and evolving nature. This limitation reduces their reliability in downstream tasks that demand genuine cultural sensitivity. In this work, we introduce CALM, a novel framework designed to endow language models with cultural self-awareness. CALM disentangles task semantics from explicit cultural concepts and latent cultural signals, shaping them into structured cultural clusters through contrastive learning. These clusters are then aligned via cross-attention to establish fine-grained interactions among related cultural features and are adaptively integrated through a Mixture-of-Experts mechanism along culture-specific dimensions. The resulting unified representation is fused with the model's original knowledge to construct a culturally grounded internal identity state, which is further enhanced through self-prompted reflective learning, enabling continual adaptation and self-correction. Extensive experiments conducted on multiple cross-cultural benchmark datasets demonstrate that CALM consistently outperforms state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。