用目标环境的文本描述提升模型泛化能力,无需目标数据。
Target-Oriented Single Domain Generalization
- 利用目标环境文本描述引导源域特征迁移。
- 在多个分类与检测任务上超越现有方法,显著提升泛化性能。
- 适合部署在缺乏目标数据但有文本描述的场景中。
在单源域训练的深度模型在分布偏移下常表现崩溃,这是单域泛化(SDG)的核心挑战。现有方法多聚焦于增强源数据或学习不变特征,却忽视了可获取的资源:目标部署环境的文本描述。我们提出目标导向的单域泛化(TO-SDG),在不需任何目标数据的前提下,利用目标域文本描述指导模型泛化。为此,我们引入光谱目标对齐(STAR),一个轻量模块,通过视觉-语言模型(如CLIP)将目标语义注入源特征。STAR基于目标描述的文本嵌入构建目标锚定子空间,将图像特征重新中心化至部署域,并通过谱投影保留与目标线索对齐的方向,同时剔除源域特异性噪声。此外,采用视觉-语言蒸馏使主干特征与VLM的语义几何对齐。STAR进一步使用特征空间Mixup,确保源与目标导向表示间平滑过渡。在多个图像分类与目标检测基准上的实验表明,该方法显著优于现有方法。本工作证明,极少的文本元数据——一种实用且常被忽略的资源——在严重数据约束下能显著提升泛化能力,为在未知数据的目标环境中部署鲁棒模型开辟新路径。
原文摘要 · Abstract (English)
Deep models trained on a single source domain often fail catastrophically under distribution shifts, a critical challenge in Single Domain Generalization (SDG). While existing methods focus on augmenting source data or learning invariant features, they neglect a readily available resource: textual descriptions of the target deployment environment. We propose Target-Oriented Single Domain Generalization (TO-SDG), a novel problem setup that leverages the textual description of the target domain, without requiring any target data, to guide model generalization. To address TO-SDG, we introduce Spectral TARget Alignment (STAR), a lightweight module that injects target semantics into source features by exploiting visual-language models (VLMs) such as CLIP. STAR uses a target-anchored subspace derived from the text embedding of the target description to recenter image features toward the deployment domain, then utilizes spectral projection to retain directions aligned with target cues while discarding source-specific noise. Moreover, we use a vision-language distillation to align backbone features with VLM's semantic geometry. STAR further employs feature-space Mixup to ensure smooth transitions between source and target-oriented representations. Experiments across various image classification and object detection benchmarks demonstrate STAR's superiority. This work establishes that minimal textual metadata, which is a practical and often overlooked resource, significantly enhances generalization under severe data constraints, opening new avenues for deploying robust models in target environments with unseen data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。