提出新框架让模型分离内容与风格,解决视觉任务中依赖纹理的投机学习问题。
Disentangling Content from Style to Overcome Shortcut Learning: A Hybrid Generative-Discriminative Learning Framework
- 通过输入扰动+不变性预训练,显式分离表征中的内容与风格成分。
- 在多个诊断投机学习的基准上超越现有方法,尤其在跨域泛化上提升显著。
- 适合关注模型鲁棒性、公平性和可解释性的研究者与工程师。
尽管自监督学习(SSL)取得了显著成功,其泛化能力仍受投机学习的根本制约——模型倾向于利用纹理等表面特征而非内在结构。我们在生成式范式(如MAE)中验证了这一缺陷,并指出其也是判别式方法的系统性问题,是其在未见领域失败的根源。现有方法多通过对齐或分离领域特定特征来应对,但未能改变催生投机依赖的学习机制。为此,我们提出HyGDL(混合生成-判别学习框架),实现内容与风格的显式解耦。该方法基于不变性预训练原则:在保持监督信号恒定的同时,系统性地改变输入偏置(如风格),迫使模型学习不变本质。HyGDL仅使用单个编码器,通过向量投影解析风格为与内容正交的表征分量。其协同设计包含三步:(1) 自蒸馏目标学习稳定的内容方向;(2) 解析投影将表征分解为正交的内容与风格向量;(3) 风格条件重建目标利用二者恢复图像,提供端到端监督。相比依赖隐式启发的先前方法,这种原理驱动的解耦使HyGDL学习到真正鲁棒的表征,在诊断投机学习的基准上表现优异。
原文摘要 · Abstract (English)
Despite the remarkable success of Self-Supervised Learning (SSL), its generalization is fundamentally hindered by Shortcut Learning, where models exploit superficial features like texture instead of intrinsic structure. We experimentally verify this flaw within the generative paradigm (e.g., MAE) and argue it is a systemic issue also affecting discriminative methods, identifying it as the root cause of their failure on unseen domains. While existing methods often tackle this at a surface level by aligning or separating domain-specific features, they fail to alter the underlying learning mechanism that fosters shortcut dependency. To address this at its core, we propose HyGDL (Hybrid Generative-Discriminative Learning Framework), a hybrid framework that achieves explicit content-style disentanglement. Our approach is guided by the Invariance Pre-training Principle: forcing a model to learn an invariant essence by systematically varying a bias (e.g., style) at the input while keeping the supervision signal constant. HyGDL operates on a single encoder and analytically defines style as the component of a representation that is orthogonal to its style-invariant content, derived via vector projection. This is operationalized through a synergistic design: (1) a self-distillation objective learns a stable, style-invariant content direction; (2) an analytical projection then decomposes the representation into orthogonal content and style vectors; and (3) a style-conditioned reconstruction objective uses these vectors to restore the image, providing end-to-end supervision. Unlike prior methods that rely on implicit heuristics, this principled disentanglement allows HyGDL to learn truly robust representations, demonstrating superior performance on benchmarks designed to diagnose shortcut learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。