揭示表征学习中协变量依赖与潜在结构的权衡关系,提升可控性。
The Trade-off Between Covariate Dependence and Latent Structure in Representation Learning

- 构建统一监督框架,联合约束潜在维度与协变量的依赖关系
- 证明保持独立性或一对一对应会降低潜在-协变量对齐精度
- 可后处理预训练模型,适配多组学等需要可控表征的场景
解耦表征学习旨在使潜在表示的各个维度分别对应一个语义协变量。无监督方法通常追求潜在维度间的独立性,但无法保证其与有意义协变量对齐;监督方法虽利用观测协变量构造潜空间,但在协变量相关时,无法同时实现一一对应和潜空间独立。本文提出统一监督框架,将潜在维度-协变量依赖与潜结构约束耦合。我们证明存在固有权衡:强制潜空间独立或严格的一一对应会导致潜在-协变量对齐性能下降。不同解耦范式按对齐强度有序排列,每种范式均有闭式潜空间变换。我们将这些变换应用于CLIP、DINOv2和ViT等预训练模型,用于重对齐表示,并集成到基于协变量信息的因子分析(iFA)中。在模拟与真实多组学数据上,后处理对齐与iFA均实现了结构化潜在表示的可控性。
原文摘要 · Abstract (English)
Disentangled representation learning seeks latent representations whose indicidual dimensions each align with a distinct covariate. Unsupervised approaches typically target latent dimension independence, yet this gives no guarantee that the resulting dimensions align with semantically meaningful covariates. Supervised approaches structure the latent space using observed covariates, but under correlated covariates they cannot simultaneously control one-to-one latent-covariate alignment and latent independence. We introduce a unified, supervised framework that couples latent dimension-covariate dependence with constraints on the latent structure. Within this framework, we show an inherent trade-off, where enforcing latent independence or exclusive one-to-one latent-covariate dependence comes at a provable cost in latent-covariate alignment. We prove that the resulting disentanglement regimes are ordered by the strength of that alignment. Each regime admits a closed-form transformation of the latent space. We apply these transformations post-hoc to realign the representations of pretrained models such as CLIP, DINOv2, and ViT, and we fold them into the inference of informed factor analysis (iFA), a probabilistic model with covariate-informed factors. On simulated and real multi-omics data, we show that both post-hoc alignment and iFA enable controllability of structured latent representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。