按区域精细分块的视觉检测新方法,大幅降低计算开销仍保持高精度。
Variable-Granularity Tokenization for High-Resolution Object Detection

- 基于局部可区分性和结构几何自动调节每块分辨率,无需训练。
- 在40%令牌预算下达44.22 AP,比现有最优高出6.08点。
- 适用于航拍小目标检测,适合资源受限场景部署。
ViT检测器在学习阶段前固定统一的令牌网格,导致高分辨率航拍检测需在识别像素级物体与控制算力内存之间权衡。本文提出VGTok,一种无需训练的预编码器令牌化方法,根据像素级区域特性动态设定分块粒度。该方法通过多尺度形态学顶帽可分性评分,并以图像级百分位阈值确定令牌预算;结构张量门(λ_min)仅在二维物体结构存在时细化,保留一维杂讯粗粒度。生成的令牌集严格划分图像。在使用EVA-02 ViT-L编码器的Co-DETR检测器中,VGTok在40%至100%令牌预算下刷新所有公开发布的VisDrone-val AP与AP_S记录。在40%预算下达到44.22 AP,丢弃五分之三序列后仍领先;密集设置达48.38 AP,超出最强已有结果6.08点。该方法迁移至AI-TOD-v2未作调整,仍达37.27 AP与19.51 AP_vt新纪录。作为冻结检查点的纯插入模块,在78.5%令牌下达36.29 AP,超越所有已有成果,且376.3M参数模型优于30亿参数多专家模型。实验表明,仅依赖局部可区分性与结构几何、在主干前固定的令牌预算,可在航拍小目标主导场景中保持精度,同时减少3.1倍编码器计算量和1.9倍内存占用。
原文摘要 · Abstract (English)
ViT detectors fix a uniform token grid before any learned stage. A native-resolution aerial detector must then choose between resolving few-pixel objects and staying inside compute and memory limits. We introduce VGTok, a training-free tokenizer that sets patch granularity per region from pixels, ahead of the encoder. VGTok scores each region by multi-scale morphological top-hat separability from its surround, then thresholds those scores at a per-image percentile, which fixes the token budget. A structure-tensor gate ($λ_{\min}$) refines only where two-dimensional object structure supports it, leaving one-dimensional clutter coarse. The resulting token set is a strict partition of the image. In a Co-DETR detector with an EVA-02 ViT-L encoder, VGTok clears every published VisDrone-val AP and AP$_S$ at every budget from 40\% to 100\% of tokens. At 40\% it records 44.22 AP with three fifths of the sequence discarded before the first transformer block; dense, it reaches 48.38 AP, $6.08$ above the strongest published entry. VGTok transfers to AI-TOD-v2 untouched, same scorer and same rank, and sets a new state of the art at 37.27 AP and 19.51 AP$_{vt}$. As a pure drop-in into a frozen checkpoint it reaches 36.29 AP at 78.5\% of tokens, above every published entry, where our 376.3M-parameter detector clears a 3.0B multi-expert model. We show that a token budget fixed before the backbone, from local separability and structure geometry alone, holds accuracy on the tiny-object regimes that dominate aerial detection, at $3.1\times$ less encoder compute and $1.9\times$ less encoder memory. Code and models are available at \href{https://github.com/khayrulbuet13/vgtok}{\texttt{github.com/khayrulbuet13/vgtok}} and \href{https://huggingface.co/khayrulbuet13/vgtok}{\texttt{huggingface.co/khayrulbuet13/vgtok}}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。