arXiv:2511.19846cs.CV2025-11

一个模型同时搞定人脸、人体和物体识别,还能避免遗忘旧知识。

Face, Whole-Person, and Object Classification in a Unified Space Via The Interleaved Multi-Domain Identity Curriculum

  • 用交错多领域身份课程训练,让模型同步学四类任务。
  • 在三个基底模型上实现四项任务全胜,精度超人类。
  • 共享特征多但任务可分,少量主成分即可完成所有任务。

视觉基础模型可在零样本模式下进行泛化物体分类,经过微调后可用于人脸/人体识别。然而,微调模型易产生灾难性遗忘。本文构建了能在单一嵌入空间中执行四项任务(物体识别、高/低质量人脸识别、全身人体识别)的模型,且几乎无灾难性遗忘。提出两种变体的交错多领域身份课程(IMIC):一种梯度耦合的交错训练策略,同时对四个任务微调基础骨干网络。IMIC方法在DINOv3、CLIP和EVA-02三种基础模型上均有效。其中两个模型(EVA-02和CLIP)在四项任务上表现与领域专家相当,且在跨人脸、身体和物体数据集的多任务场景中优于人类。进一步证明该方法未显著损害分布外泛化能力,保持了基础模型的核心优势。对最准确模型(EVA-02 + IMIC A/B)的分析显示,四类任务在统一嵌入空间中线性可分,但存在大量特征共享。仅需少于100个主成分(PCs)即可完成其他任务,性能损失近乎为零。

原文摘要 · Abstract (English)

Vision foundation models can perform generalized object classification in zero-shot mode, and face/person recognition when they are fine-tuned. However, fine-tuned models suffer from catastrophic forgetting. We create models that perform four tasks (object recognition, face recognition from high- and low-quality images, and person recognition from whole-body images) in a single embedding space -- without incurring substantial catastrophic forgetting. To accomplish this, we introduce two variants of the Interleaved Multi-Domain Identity Curriculum (IMIC): a gradient-coupled, interleaving training schedule that fine-tunes a foundation backbone simultaneously on all four tasks. The IMIC method proved effective with three foundation model bases: DINOv3, CLIP, and EVA-02. Two of these (EVA-02 and CLIP) performed comparably with domain experts on all four tasks concurrently and were more accurate than humans at multi-tasking across face, body, and object datasets. Further, we demonstrate that our approach does not substantially harm out-of-distribution generalization, thus maintaining a key property of foundation models. Analysis of the most accurate model variants (EVA-02 + IMIC A and B) showed linearly separable representations of the four tasks in the unified embedding space, but with substantial sharing of features across tasks. Fewer than 100 PCs calculated from any one task could perform all other tasks with nearly zero performance degradation.

多任务学习特征共享模型泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。