arXiv:2604.03428cs.CVcs.AI2026-04

不微调模型,用路径复制提升海洋物种分类精度

Inference-Path Optimization via Circuit Duplication in Frozen Visual Transformers for Marine Species Classification

  • 推理时复制部分Transformer层路径,实现无训练优化
  • 最优情况下宏F1达0.875,接近全监督模型表现
  • 多数物种受益于专属路径,适合标签稀缺场景

自动化水下物种分类受限于标注成本和环境变化导致的模型泛化性不足。近期研究表明,自监督视觉基础模型的冻结嵌入已能提供高效的标签高效基准。本文研究是否可在不微调或修改模型权重的前提下,在推理阶段改进这一冻结嵌入方案。我们引入原本用于大语言模型的电路复制(Circuit Duplication)方法:在前向传播中对选定的一组Transformer层进行两次遍历。在类别不平衡的AQUA20基准上,使用冻结的DINOv3嵌入,分别测试全局电路选择(全数据集共用一个复制路径)与类别特异性电路选择(每类可选不同最优路径)。两者均采用简单的半监督下游分类器。实验表明,电路复制显著优于标准冻结前向传播。在最大标签预算下,类别特异性选择达到宏F1 0.875,与全监督ConvNeXt基准(0.889)差距缩小至1.4点,且无需任何梯度训练;其中八种物种超越其全监督参考值,章鱼提升+12.1 F1点。所有预算下约75%类别偏好专属电路,证明存在真实类别依赖收益。据我们所知,这是电路复制首次应用于计算机视觉。

原文摘要 · Abstract (English)

Automated underwater species classification is constrained by annotation cost and environmental variation that limits the transferability of fully supervised models. Recent work has shown that frozen embeddings from self-supervised vision foundation models already provide a strong label-efficient baseline for marine image classification. Here we investigate whether this frozen-embedding regime can be improved at inference time, without fine-tuning or changing model weights. We apply Circuit Duplication, an inference-time method originally proposed for Large Language Models, in which a selected range of transformer layers is traversed twice during the forward pass. We evaluate on the class-imbalanced AQUA20 benchmark using frozen DINOv3 embeddings under two settings: global circuit selection, where a single duplicated circuit is chosen for the full dataset, and class-specific circuit selection, where each species may receive a different optimal circuit. Both settings use simple semi-supervised downstream classifiers. Circuit Duplication consistently improves over the standard frozen forward pass. At the maximum label budget, class-specific selection reaches a macro F1 of 0.875, closing the gap to the fully supervised ConvNeXt benchmark (0.889) to 1.4 points without any gradient-based training. Four species exceed their fully supervised reference, with octopus improving by +12.1 F1 points. Across all budgets, roughly 75% of classes prefer a class-specific circuit, indicating a genuinely class-dependent benefit. To our knowledge, this is the first application of Circuit Duplication to computer vision.

视觉模型推理优化海洋分类冻结嵌入

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。