提出新适配器EGA，让冻结视觉模型在未知类别上搜索更稳定。
EGA: Adapting Frozen Encoders for Vector Search with Bounded Out-of-Distribution Degradation
- 用局部三元组损失+球面投影，实现自限制更新机制
- 96.5%三元组停止更新，未知类区域几乎不变
- 在5个外部数据集上显著提升最差情况下的标签精度
基于冻结视觉编码器的向量搜索系统在部署时会遇到未见类别查询，现有适配器训练在此分布外（OOD）场景下会失效：高容量适配器配合全局对比损失会悄然将未见类别样本错误分配到已有类别簇中，导致最差情况下的标签精度比冻结基线下降超过40点。本文提出欧几里得测地线对齐（EGA），一种残差适配器，融合零初始化、局部三元组损失与超球面投影三个原则。该设计诱导出自限动态：已满足小间距条件的三元组停止产生梯度，使适配器在局部几何已正确处自动停止更新。实验显示，收敛时96.5%的三元组为无梯度状态，未见类别区域基本不受影响，同时仍可充分优化已见类别。在五个多样化的外部数据集基准上，EGA在四个主要分割上取得最高最差标签精度，并在第五个上保持一致提升。该设计还可扩展至CLIP以外的更强骨干网络，我们还提供了梯度稀疏性与受限OOD扰动之间的分析证明。
原文摘要 · Abstract (English)
Vector search systems built on frozen vision encoders face queries from unseen classes at deployment, yet existing adapter training collapses under this shift: high-capacity adapters with global contrastive losses silently reassign unseen-class samples to wrong seen-class clusters, dropping worst-case Label Precision by over 40 points below the frozen baseline in our tests. We propose Euclidean Geodesic Alignment (EGA), a residual adapter that couples three principles: zero initialization, local triplet loss, and hypersphere projection. These collectively induce a self-limiting dynamic: triplets that already satisfy a small margin stop producing gradients, so the adapter automatically stops updating where the local geometry is already correct. Our experiments show that at convergence $96.5\%$ of triplets are gradient-free, leaving unseen-class regions largely untouched while still enabling full-capacity refinement of seen classes. Across five diverse out-of-distribution (OOD) benchmarks, EGA achieves the highest worst-case Label Precision on the four primary splits and a consistent improvement on the fifth. The design also transfers to stronger backbones in addition to CLIP, and we provide an analytical justification linking gradient sparsity to bounded OOD perturbation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。