arXiv:2512.12498cs.CV2025-12

用图像块间关系优化缓存适配,提升少样本分类准确率

Advancing Cache-Based Few-Shot Classification via Patch-Driven Relational Gated Graph Attention

  • 基于图像块间关系构建图注意力网络,动态生成适配权重
  • 在11个基准上超越现有方法,显著提升少样本分类性能
  • 适用于无人机救援等时间敏感场景的快速判伤任务

少样本图像分类在监督数据有限和视觉领域偏移下仍具挑战。现有基于缓存的自适应方法(如Tip-Adapter)通过在冻结特征上学习轻量级残差适配器缓解问题,但仍继承CLIP全局通用表征的倾向,难以在低数据条件下将通用模型适配至特定领域。本文提出一种新的基于图像块的关联性精炼方法,不再将图像嵌入视为单一向量,而是从图像块内部依赖关系中学习缓存适配权重。具体而言,引入关系门控图注意力网络,构建块图并执行边感知注意力,强调关键块间交互,生成上下文增强的块嵌入;可学习的多聚合池化将其整合为紧凑、任务判别性强的表示,更匹配目标少样本类别。关键在于,该图精炼仅用于训练阶段,将关系结构蒸馏至缓存,推理时仅需标准缓存查找,无额外开销。最终预测通过缓存相似度与CLIP零样本逻辑值的残差融合获得。在11个基准上广泛评估显示,性能持续优于最先进CLIP适配器与缓存基线,同时保持零样本效率。进一步通过引入‘受伤/未受伤士兵’数据集验证战场适用性,旨在支持‘黄金分钟’内的时间敏感无人机搜救与战伤救治中的分诊决策。

原文摘要 · Abstract (English)

Few-shot image classification remains difficult under limited supervision and visual domain shift. Recent cache-based adaptation approaches (e.g., Tip-Adapter) address this challenge to some extent by learning lightweight residual adapters over frozen features, yet they still inherit CLIP's tendency to encode global, general-purpose representations that are not optimally discriminative to adapt the generalist to the specialist's domain in low-data regimes. We address this limitation with a novel patch-driven relational refinement that learns cache adapter weights from intra-image patch dependencies rather than treating an image embedding as a monolithic vector. Specifically, we introduce a relational gated graph attention network that constructs a patch graph and performs edge-aware attention to emphasize informative inter-patch interactions, producing context-enriched patch embeddings. A learnable multi-aggregation pooling then composes these into compact, task-discriminative representations that better align cache keys with the target few-shot classes. Crucially, the proposed graph refinement is used only during training to distil relational structure into the cache, incurring no additional inference cost beyond standard cache lookup. Final predictions are obtained by a residual fusion of cache similarity scores with CLIP zero-shot logits. Extensive evaluations on 11 benchmarks show consistent gains over state-of-the-art CLIP adapter and cache-based baselines while preserving zero-shot efficiency. We further validate battlefield relevance by introducing an Injured vs. Uninjured Soldier dataset for casualty recognition. It is motivated by the operational need to support triage decisions within the "platinum minutes" and the broader "golden hour" window in time-critical UAV-driven search-and-rescue and combat casualty care.

少样本学习图神经网络缓存适配无人机救援

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。