arXiv:2604.15494cs.LGcs.CV2026-04

通过原型信号提升模型在分布偏移下的鲁棒性,保持可解释性。

ProtoTTA: Prototype-Guided Test-Time Adaptation

论文配图:ProtoTTA: Prototype-Guided Test-Time Adaptation
图 1 · 摘自论文原文
  • 利用中间原型信号优化测试时适应,而非仅依赖输出。
  • 降低原型相似度分布熵,使激活更聚焦且可信。
  • 适合医疗、视觉等需可解释性的高风险场景。

基于原型的深度网络因兼具高精度与可解释性,被广泛应用于医疗等关键领域。然而其性能受限于训练数据,对分布偏移敏感。现有测试时适应(TTA)方法主要更新参数和统计量,未充分考虑原型信息。本文提出ProtoTTA,一种通用框架,通过中间原型信号引导适应过程。该方法最小化原型相似度分布的熵,增强在分布偏移数据上的原型特异性激活。为保证稳定性,采用几何过滤机制,仅允许原型激活可靠的样本进行更新,并结合原型重要性权重与模型置信度正则化。在四个不同基准(涵盖细粒度视觉、组织病理学、NLP)上使用四种原型骨干模型的实验表明,ProtoTTA优于标准输出熵最小化,在恢复正确语义焦点方面表现更优。此外,引入新型可解释性指标与视觉-语言模型(VLM)评估框架,验证了ProtoTTA能恢复人类对齐的语义关注,并与VLM评估的推理质量高度相关。代码已开源:https://github.com/DeepRCL/ProtoTTA。

原文摘要 · Abstract (English)

Deep networks that rely on prototypes-interpretable representations that can be related to the model input-have gained significant attention for balancing high accuracy with inherent interpretability, which makes them suitable for critical domains such as healthcare. However, these models are limited by their reliance on training data, which hampers their robustness to distribution shifts. While test-time adaptation (TTA) improves the robustness of deep networks by updating parameters and statistics, the prototypes of interpretable models have not been explored for this purpose. We introduce ProtoTTA, a general framework for prototypical models that leverages intermediate prototype signals rather than relying solely on model outputs. ProtoTTA minimizes the entropy of the prototype-similarity distribution to encourage more confident and prototype-specific activations on shifted data. To maintain stability, we employ geometric filtering to restrict updates to samples with reliable prototype activations, regularized by prototype-importance weights and model-confidence scores. Experiments across four prototypical backbones on four diverse benchmarks spanning fine-grained vision, histopathology, and NLP demonstrate that ProtoTTA improves robustness over standard output entropy minimization while restoring correct semantic focus in prototype activations. We also introduce novel interpretability metrics and a vision-language model (VLM) evaluation framework to explain TTA dynamics, confirming ProtoTTA restores human-aligned semantic focus and correlates reliably with VLM-rated reasoning quality. Code is available at: https://github.com/DeepRCL/ProtoTTA.

原型学习测试时适应可解释性医学图像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。