让AI模型学会区分因果与虚假关联,提升部署时的稳定性。
Causal Fine-Tuning under Latent Confounded Shift
- 基于因果模型分解表示,分离稳定与易变特征
- 在文本任务中显著优于现有领域泛化方法
- 适合需要应对隐藏干扰变量的现实场景
现代人工智能在适应隐含混淆偏移方面仍面临核心挑战。该问题由训练阶段隐藏变量引发的输入与输出间虚假相关性导致,使模型依赖非因果捷径。例如,模型可能将数据来源(如“Amazon”)作为正面情感的代理,在部署时因来源转为负面而失效。为此,我们提出因果微调(Causal Fine-Tuning, CFT),利用结构因果模型作为归纳偏置,推导出充分识别条件,并设计微调目标,将表示分解为高层稳定成分与低层敏感成分。在BERT上实现该框架后,学习并调整因果与虚假表示,可获得更鲁棒的预测器。在文本中注入虚假相关性的攻击实验表明,本方法显著优于黑盒领域泛化基线,凸显显式建模因果结构的优势。
原文摘要 · Abstract (English)
Adapting to latent confounded shift remains a core challenge in modern AI. This setting is driven by hidden variables that induce spurious correlations between inputs and outputs during training, leading models to rely on non-causal shortcuts. For example, a model may learn to treat metadata (e.g., data source like "Amazon") as a proxy for positive sentiment, causing failure when the source becomes predominantly negative during deployment. To address this latent confounded shift, we introduce Causal Fine-Tuning(CFT). Using a structural causal model as an inductive bias, we derive sufficient identification conditions that motivate a fine-tuning objective for decomposing representations into high-level stable and low-level shift-sensitive components. Instantiating this framework in BERT, we show that learning such causal/spurious representations and adjusting them accordingly yield a more robust predictor. Experiments on spurious correlation injection attacks in text demonstrate that our method outperforms black-box domain generalization baselines, highlighting the benefits of explicitly modeling causal structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。