分离真实信号与虚假相关,提升模型在复杂数据中的泛化能力。
Supervised Contrastive Block Disentanglement
- 通过监督对比学习,分别建模目标信号与环境干扰。
- 在域泛化和单细胞批次校正任务中显著提升外分布性能。
- 仅需调节一个超参数即可控制对环境变化的不变性程度。
现实数据常混合不同实验条件下的样本,虽扩大了数据规模,却引入虚假相关,阻碍对核心现象的建模。本文提出监督对比块解纠缠(SCBD)算法,学习两个独立嵌入:一个与目标变量 $y$ 相关且对环境变量 $e$ 不变,另一个则反映 $e$ 的虚假关联。该方法基于监督对比学习,有效实现对 $e$ 的不变性,在真实数据上优于现有方法。我们在两个挑战性任务中验证:一是领域泛化,在合成数据集和 Camelyon17-WILDS 上表现优异;二是批次校正,应用于2600万张光学池筛选图像,成功保留生物信号并去除跨孔批次效应。引入单一超参数 $α$ 控制不变性强度,增大 $α$ 可提升外分布性能,但牺牲内分布表现。
原文摘要 · Abstract (English)
Real-world datasets often combine data collected under different experimental conditions. This yields larger datasets, but also introduces spurious correlations that make it difficult to model the phenomena of interest. We address this by learning two embeddings to independently represent the phenomena of interest and the spurious correlations. The embedding representing the phenomena of interest is correlated with the target variable $y$, and is invariant to the environment variable $e$. In contrast, the embedding representing the spurious correlations is correlated with $e$. The invariance to $e$ is difficult to achieve on real-world datasets. Our primary contribution is an algorithm called Supervised Contrastive Block Disentanglement (SCBD) that effectively enforces this invariance. It is based purely on Supervised Contrastive Learning, and applies to real-world data better than existing approaches. We empirically validate SCBD on two challenging problems. The first problem is domain generalization, where we achieve strong performance on a synthetic dataset, as well as on Camelyon17-WILDS. We introduce a single hyperparameter $α$ to control the degree of invariance to $e$. When we increase $α$ to strengthen the degree of invariance, out-of-distribution performance improves at the expense of in-distribution performance. The second problem is batch correction, in which we apply SCBD to preserve biological signal and remove inter-well batch effects when modeling single-cell perturbations from 26 million Optical Pooled Screening images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。