用48字节存句子嵌入，32倍压缩且无需训练
Clark Hash: Stateless Sparse Johnson-Lindenstrauss Quantization for Neural Embeddings
- 无状态稀疏投影+量化编码，直接压缩嵌入向量
- 48字节存1536字节的嵌入，压缩比达32倍
- 适合需要快速存储/检索的多语言相似度任务
Clark Hash是一种用于高效存储神经嵌入的小型方法。它对每个数据库向量进行归一化，应用确定性的稀疏符号约翰逊-林登斯特拉乌斯投影，截断结果，并存储固定宽度的标量量化码。查询保持浮点格式，与存储的摘要进行余弦相似度计算。在默认的384维句嵌入设置下，克拉克哈希将一个余弦搜索向量压缩至48字节，相比密集f32存储的1536字节减少了32倍。该方法无需训练、无需学习码本、无需旋转或语料库统计即可存储新向量。我们描述了编解码器、Rust实现，并在29个子集共9,304个标注对上评估了多语言句相似性。使用多语言MiniLM编码器，48字节摘要在STS17和STS22上分别达到0.910和0.946的宏皮尔逊相关系数。克拉克哈希并非新的约翰逊-林登斯特拉乌斯定理，也不是近似最近邻索引的替代品，而是一种简单的无状态嵌入压缩编码器。
原文摘要 · Abstract (English)
Clark Hash is a small method for storing neural embeddings in less space. It normalizes each database vector, applies a deterministic sparse signed Johnson-Lindenstrauss projection, clips the result, and stores a fixed-width scalar-quantized code. Queries stay in floating point and are scored against the stored sketches. In the default 384-dimensional sentence-embedding setting, Clark Hash stores a cosine-search vector in 48 bytes instead of 1536 bytes for dense f32 storage. This is 32x smaller. The method does not need a training pass, learned codebooks, rotations, or corpus statistics before new vectors can be stored. We describe the codec, the Rust implementation, and a multilingual sentence-similarity evaluation on 9,304 labeled pairs from 29 subsets. With a multilingual MiniLM encoder, the 48-byte sketches reached 0.910 and 0.946 macro Pearson correlation with dense cosine scores on STS17 and STS22. Clark Hash is not a new Johnson-Lindenstrauss theorem and it is not a replacement for approximate nearest-neighbor indexes. It is a simple stateless codec for compact embedding storage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。