提出信息相减框架，可分离连续变量间的条件熵与互信息。
Information Subtraction: Learning Representations for Conditional Entropy
- 通过生成式架构同时最大化与最小化信息项，实现信息分离。
- 在公平学习与域泛化任务中表现优异，有效去除敏感与领域特异性信息。
- 支持迭代操作，可解析任意连续变量间的信息成分，适合因果分析场景。
条件熵与条件互信息的表征对解释变量间的独特效应至关重要。尽管基于条件对比采样的先前研究已有效移除离散敏感变量的信息，但尚未扩展至连续情形。本文提出信息相减框架，旨在生成保留所需信息而消除无关信息的表征。我们采用生成式架构，通过同时最大化一个信息项并最小化另一个来输出此类表征。凭借其在信息解耦上的灵活性，可迭代应用该方法以表示连续变量间任意信息成分，从而揭示其复杂关系。结果表明，该表征能提供条件熵的语义特征。通过移除敏感与领域特定信息，本框架在公平学习与域泛化任务中表现出色。代码已公开于 https://github.com/jh-liang/Information-Subtraction。
原文摘要 · Abstract (English)
The representations of conditional entropy and conditional mutual information are significant in explaining the unique effects among variables. While previous studies based on conditional contrastive sampling have effectively removed information regarding discrete sensitive variables, they have not yet extended their scope to continuous cases. This paper introduces Information Subtraction, a framework designed to generate representations that preserve desired information while eliminating the undesired. We implement a generative-based architecture that outputs these representations by simultaneously maximizing an information term and minimizing another. With its flexibility in disentangling information, we can iteratively apply Information Subtraction to represent arbitrary information components between continuous variables, thereby explaining the various relationships that exist between them. Our results highlight the representations' ability to provide semantic features of conditional entropy. By subtracting sensitive and domain-specific information, our framework demonstrates effective performance in fair learning and domain generalization. The code for this paper is available at https://github.com/jh-liang/Information-Subtraction
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。