arXiv:2607.03328cs.CVcs.IR2026-07

提出HashViT,让ViT直接生成可检索的二进制编码,提升图像检索效率与精度。

Beyond Post-Quantization: Native Hash Learning with a Dedicated HASH Token

论文配图:Beyond Post-Quantization: Native Hash Learning with a Dedicated HASH Token
图 1 · 摘自论文原文
  • 在ViT中引入专用哈希令牌,实现从特征到二进制码的渐进式演化。
  • 在三个基准上达到领先或接近顶尖的检索性能,保持哈希码高效性。
  • 适合需要高效大规模图像检索的工程应用,如视觉搜索、推荐系统。

高效的大规模图像检索需要紧凑的表示,以在快速的汉明空间搜索中保留语义相似性。深度哈希具有吸引力,但大多数基于CNN和ViT的方法仍遵循后量化范式:先学习连续视觉特征,再通过终端哈希投影或二值化操作生成二进制码。这种后期码生成导致表示空间与检索使用的离散汉明空间之间存在特征-码不一致问题。为此,我们提出HashViT,一种用于原生哈希标记学习的视觉变换器框架。不同于将哈希视为末端读出,HashViT引入一个专用的HASH令牌,作为变压器内部持续存在的哈希导向检索状态。该令牌结构分解为哈希寄存器(直接生成二进制码)和语义工作区(保留辅助连续语义)。为实现有效的工作区-寄存器交互,我们进一步设计了一个轻量级哈希精炼适配器,在变压器层间逐步精炼哈希寄存器。结果是,二进制导向表示通过骨干网络内的标记演化形成,而非由输出层投影突然诱导。HashViT采用统一目标进行优化,包含可学习语义中心监督、类别令牌相似性蒸馏和量化正则化,促使HASH令牌编码出语义结构化且紧凑的二进制表示。在三个广泛使用的基准上的大量实验表明,HashViT实现了最先进的或极具竞争力的检索性能,同时保持紧凑汉明码的高效性。代码已公开于 https://github.com/Xinze919/HashViT。

原文摘要 · Abstract (English)

Efficient large-scale image retrieval requires compact representations that preserve semantic similarity under fast Hamming-space search. Deep hashing is appealing, but most existing CNN- and ViT-based methods still follow a post-quantization paradigm, where continuous visual features are first learned and binary codes are then produced by a terminal hash projection or binarization operation. This late code generation creates a feature-to-code discrepancy between the continuously optimized representation space and the discrete Hamming space used for retrieval. To address this limitation, we propose HashViT, a Vision Transformer framework for native hash token learning. Instead of treating hashing as a terminal readout, HashViT introduces a dedicated HASH token that serves as a persistent, hash-oriented retrieval state inside the transformer. The HASH token is structurally decomposed into a Hash Register for direct binary code generation and a Semantic Workspace for preserving auxiliary continuous semantics. To enable effective workspace-to-register interaction, we further design a lightweight Hash Refinement Adapter that progressively refines the Hash Register across transformer layers. As a result, binary-oriented representations are formed through token evolution within the backbone, rather than being abruptly induced by an output-level projection. HashViT is optimized with a unified objective that combines learnable semantic center supervision, class-token similarity distillation, and quantization regularization, encouraging the HASH token to encode semantically structured and compact binary representations. Extensive experiments on three widely used benchmarks demonstrate that HashViT achieves state-of-the-art or highly competitive retrieval performance while preserving the efficiency of compact Hamming codes. Code is available at https://github.com/Xinze919/HashViT.

哈希检索ViT二进制编码图像搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。