用统一方法生成高效二进制哈希码,16位下3分钟内完成训练。
Image Hashing via Cross-View Code Alignment in the Age of Foundation Models
- 通过跨视图代码对齐,仅用一个交叉熵损失实现二进制码一致。
- 16位哈希在COCO上无监督训练<2分钟,ImageNet100有监督训练约3分钟。
- 可直接用于冻结嵌入或LoRA微调,适配多种场景与模型。
高效的大规模检索需要紧凑且具有区分性的表示。基础模型提供强大的视觉和多模态嵌入,但在高维空间中进行最近邻搜索计算开销大。哈希通过二进制码实现快速汉明距离搜索,但现有方法常依赖复杂流程、多目标优化、特定学习范式设计及长时间训练。本文提出CroVCA(Cross-View Code Alignment),一种简单统一的二进制码学习原则,确保语义对齐视图间代码一致性。单一二进制交叉熵损失强制对齐,编码率最大化作为防坍缩正则项以促进平衡多样代码。为此设计HashCoder,一个轻量级MLP哈希网络,末尾添加批量归一化层以保证代码均衡。HashCoder可作为探测头应用于冻结嵌入,或通过LoRA微调高效适配编码器。在多个基准测试中,CroVCA仅需5个训练轮次即达当前最优表现。16位时尤其出色:COCO无监督哈希训练时间低于2分钟,ImageNet100有监督哈希约3分钟(单GPU)。结果凸显CroVCA的高效性、可适应性和广泛适用性。
原文摘要 · Abstract (English)
Efficient large-scale retrieval requires representations that are both compact and discriminative. Foundation models provide powerful visual and multimodal embeddings, but nearest neighbor search in these high-dimensional spaces is computationally expensive. Hashing offers an efficient alternative by enabling fast Hamming distance search with binary codes, yet existing approaches often rely on complex pipelines, multi-term objectives, designs specialized for a single learning paradigm, and long training times. We introduce CroVCA (Cross-View Code Alignment), a simple and unified principle for learning binary codes that remain consistent across semantically aligned views. A single binary cross-entropy loss enforces alignment, while coding-rate maximization serves as an anti-collapse regularizer to promote balanced and diverse codes. To implement this, we design HashCoder, a lightweight MLP hashing network with a final batch normalization layer to enforce balanced codes. HashCoder can be used as a probing head on frozen embeddings or to adapt encoders efficiently via LoRA fine-tuning. Across benchmarks, CroVCA achieves state-of-the-art results in just 5 training epochs. At 16 bits, it performs particularly well; for instance, unsupervised hashing on COCO completes in under 2 minutes and supervised hashing on ImageNet100 in about 3 minutes on a single GPU. These results highlight CroVCA's efficiency, adaptability, and broad applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。