通过动态屏蔽关键错误特征,提升模型跨领域情感分类稳定性。
Attribution-Guided Masking for Robust Cross-Domain Sentiment Classification

- 训练时根据梯度归因自动识别并惩罚易混淆的领域特异性词元。
- 在最难的跨域任务上性能超越多个强基线,零样本迁移下误差仅差0.244。
- 无需目标域标签或人工标注,且可解释哪些词元影响泛化效果。
预训练Transformer模型在本域情感分类中表现优异,但在跨域数据上常出现严重性能下降。我们假设这一泛化差距源于对领域特异性虚假词元的依赖。尽管事后词元级归因漂移无法预测该差距,我们提出训练期干预方法Attribution-Guided Masking(AGM),通过基于梯度的归因屏蔽损失($/mathcal{L}_{mask}$)动态检测并惩罚高归因的虚假词元。该方法可选地结合反事实对比损失以强化领域不变表示,且无需目标域标签或人工标注。在四个不同领域、八组随机种子的严格零样本迁移设置下评估,AGM在最困难的跨域任务Sentiment140上表现优于五种强基线:Δ = 0.244,优于DANN(0.264)、DRO(0.248)、Fish(0.247)和IRM(0.238)。同时,其提供词元级可解释性,揭示驱动泛化差距的关键特征。定性分析表明,AGM抑制了@提及、话题标签和俚语等领域的归因,转而依赖领域不变的情感标记。消融实验进一步证实归因引导屏蔽是核心:移除或替换为随机选择会持续降低困难迁移任务上的性能。
原文摘要 · Abstract (English)
While pre-trained Transformer models achieve high accuracy on in-domain sentiment classification, they frequently experience severe performance degradation when transferring to out-of-domain data. We hypothesize that this generalization gap is driven by reliance on domain-specific spurious tokens. After demonstrating that post-hoc-token-level attribution drift fails to predict this gap, we propose Attribution-Guided Masking (AGM), a training time intervention that dynamically detects and penalizes highly attributed spurious tokens during fine-tuning. AGM's core component is a gradient based attribution masking loss ($\mathcal{L}_{mask}$), which can optionally be combined with a counterfactual contrastive loss to enforce domain-invariant representations, all without requiring target-domain labels or human annotation. Evaluated in a strict zero-shot transfer setting across four diverse domains with eight random seeds, AGM achieves competitive generalization compared to five strong baselines on the hardest transfer (Sentiment140): $Δ$ = 0.244 versus DANN (0.264), DRO (0.248), Fish (0.247), and IRM (0.238), while uniquely providing token-level interpretability into which features drive the generalization gap. Our qualitative analysis confirms that AGM suppresses attribution on domain-specific tokens such as @mentions, hashtags, and slang, shifting reliance toward domain-invariant sentiment markers. Our ablation study further confirms that attribution-guided masking is the critical component: removing it or replacing it with random token selection consistently degrades performance on difficult transfers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。