冻结特征提取器,通过优化决策边界实现高效可解释的领域自适应。
Feature-Space Planes Searcher: A Universal Domain Adaptation Framework for Interpretability and Computational Efficiency
- 基于特征空间几何结构,仅优化决策边界而非微调模型。
- 在多个数据集上性能媲美甚至超越现有方法,计算开销大幅降低。
- 适合需要快速部署、高可解释性的实际应用,如遥感与生物预测。
领域偏移导致模型从有标签源域迁移到无标签目标域时性能下降,是深度学习系统部署中的长期挑战。现有无监督领域自适应(UDA)方法多依赖微调特征提取器,存在效率低、可解释性差、难扩展至现代架构等问题。我们分析发现,大规模预训练模型在特征空间中呈现类内紧凑、类间分离的域不变几何模式,表明领域偏移主要表现为边界错位而非特征退化。因此,我们提出特征空间平面搜索器(FPS):一种新框架,利用这些几何特性优化决策边界,同时冻结特征编码器。该方法通过离线特征提取实现单次计算周期内的全数据集优化,显著降低内存与计算成本,且支持可解释性分析。在公开基准测试中,FPS表现优于或媲美当前最优方法,并在蛋白质结构预测、遥感分类、地震检测等多领域展现泛化能力。我们预期其将成为迁移学习,尤其是领域自适应任务中的通用高效范式。
原文摘要 · Abstract (English)
Domain shift, characterized by degraded model performance during transition from labeled source domains to unlabeled target domains, poses a persistent challenge for deploying deep learning systems. Current unsupervised domain adaptation (UDA) methods predominantly rely on fine-tuning feature extractors - an approach limited by inefficiency, reduced interpretability, and poor scalability to modern architectures. Our analysis reveals that models pretrained on large-scale data exhibit domain-invariant geometric patterns in their feature space, characterized by intra-class clustering and inter-class separation, thereby preserving transferable discriminative structures. These findings indicate that domain shifts primarily manifest as boundary misalignment rather than feature degradation. Unlike fine-tuning entire pre-trained models - which risks introducing unpredictable feature distortions - we propose the Feature-space Planes Searcher (FPS): a novel domain adaptation framework that optimizes decision boundaries by leveraging these geometric patterns while keeping the feature encoder frozen. This streamlined approach enables interpretative analysis of adaptation while substantially reducing memory and computational costs through offline feature extraction, permitting full-dataset optimization in a single computation cycle. Evaluations on public benchmarks demonstrate that FPS achieves competitive or superior performance to state-of-the-art methods. FPS scales efficiently with multimodal large models and shows versatility across diverse domains including protein structure prediction, remote sensing classification, and earthquake detection. We anticipate FPS will provide a simple, effective, and generalizable paradigm for transfer learning, particularly in domain adaptation tasks. .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。