提出最小充分语义表示,无需标签实现更强的无监督域泛化。
Minimal Semantic Sufficiency Meets Unsupervised Domain Generalization
- 从信息论出发,定义语义充分且最小的表示目标
- 在多个基准上超越现有方法,无需类别或域标签
- 适合研究无监督学习与域泛化的新手和进阶者
深度学习的泛化能力在有监督场景下已得到广泛研究,但在无监督场景中仍较少被探索。近期提出的无监督域泛化(UDG)任务旨在提升基于自监督学习(SSL)等技术训练模型的泛化性能。UDG面临的关键挑战是在无类别标签的情况下区分语义与变化。尽管部分方法使用域标签解决该问题,但实际场景中这些标签往往不可用。本文将UDG形式化为学习最小充分语义表示的任务:该表示需(i)保留跨增强视图共享的所有语义信息(充分性),并(ii)最大程度去除与语义无关的信息(最小性)。我们从信息论角度理论证明,优化表示以达成充分性与最小性可直接降低分布外风险。实践中,我们通过最小充分无监督域泛化(MS-UDG)实现该优化,其包含:(a) 基于InfoNCE的目标以实现充分性;(b) 两种互补组件促进最小性:一种新颖的语义-变化解耦损失,以及基于重建的机制以捕捉充分变化。实验表明,MS-UDG在主流无监督域泛化基准上达到新最优,始终优于现有SSL与UDG方法,且表示学习阶段无需类别或域标签。
原文摘要 · Abstract (English)
The generalization ability of deep learning has been extensively studied in supervised settings, yet it remains less explored in unsupervised scenarios. Recently, the Unsupervised Domain Generalization (UDG) task has been proposed to enhance the generalization of models trained with prevalent unsupervised learning techniques, such as Self-Supervised Learning (SSL). UDG confronts the challenge of distinguishing semantics from variations without category labels. Although some recent methods have employed domain labels to tackle this issue, such domain labels are often unavailable in real-world contexts. In this paper, we address these limitations by formalizing UDG as the task of learning a Minimal Sufficient Semantic Representation: a representation that (i) preserves all semantic information shared across augmented views (sufficiency), and (ii) maximally removes information irrelevant to semantics (minimality). We theoretically ground these objectives from the perspective of information theory, demonstrating that optimizing representations to achieve sufficiency and minimality directly reduces out-of-distribution risk. Practically, we implement this optimization through Minimal-Sufficient UDG (MS-UDG), a learnable model by integrating (a) an InfoNCE-based objective to achieve sufficiency; (b) two complementary components to promote minimality: a novel semantic-variation disentanglement loss and a reconstruction-based mechanism for capturing adequate variation. Empirically, MS-UDG sets a new state-of-the-art on popular unsupervised domain-generalization benchmarks, consistently outperforming existing SSL and UDG methods, without category or domain labels during representation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。