让复杂生成模型学会因果解耦表示，支持分布外生成。
Intervening to Learn and Compose Causally Disentangled Representations
- 通过添加上下文模块，隐式逆向线性表示来学习概念信息。
- 在真实与模拟数据上实现可组合的因果解耦表示，支持分布外生成。
- 无需重构模型即可端到端训练或微调，适合需要可控生成的场景。
在设计生成模型时，通常认为表达能力与结构之间存在根本矛盾。本文挑战这一观点，提出一种新方法，可在任意复杂的黑箱模型中同时学习因果解耦的概念表示。该方法通过添加一个简单上下文模块，使模型在训练中通过隐式逆向编码器的线性表示来处理概念信息。受因果干预启发，该模块在训练中动态调整自身架构，从而学习不同上下文下的紧凑联合模型。实验表明，该方法可生成可组合的因果解耦表示，适用于真实与模拟数据上的分布外生成。模型支持端到端训练或从预训练模型微调。此外，本文还证明了一个新的可识别性结果，扩展了现有对结构化表示识别的研究。
原文摘要 · Abstract (English)
In designing generative models, it is commonly believed that in order to learn useful latent structure, we face a fundamental tension between expressivity and structure. In this paper we challenge this view by proposing a new approach to training arbitrarily expressive generative models that simultaneously learn causally disentangled concepts. This is accomplished by adding a simple context module to an arbitrarily complex black-box model, which learns to process concept information by implicitly inverting linear representations from the model's encoder. Inspired by the notion of intervention in a causal model, our module selectively modifies its architecture during training, allowing it to learn a compact joint model over different contexts. We show how adding this module leads to causally disentangled representations that can be composed for out-of-distribution generation on both real and simulated data. The resulting models can be trained end-to-end or fine-tuned from pre-trained models. To further validate our proposed approach, we prove a new identifiability result that extends existing work on identifying structured representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。