提出自适应谱温调技术，让向量压缩更高效且无需训练。
Spectral Tempering for Embedding Compression in Dense Passage Retrieval
- 根据信号噪声比动态调整降维权重，不依赖固定超参
- 在不同维度下逼近网格搜索最优性能，误差小于1%
- 无需标注数据或调参，适用于各类检索模型
维度压缩对大规模密集段落检索系统部署至关重要。主流后处理方法存在根本权衡：主成分分析（PCA）保留主要方差但浪费表征能力，而白化虽强制各向同性却放大重尾特征谱中的噪声。中间谱缩放方法通过幂系数γ重加权各维度，但通常将γ设为固定超参，需针对任务调优。我们发现最优缩放强度γ并非全局常数，其随目标维度k变化，由保留子空间的信噪比（SNR）决定。基于此，我们提出无学习的Spectral Tempering（SpecTemp）方法，仅通过局部SNR分析与拐点归一化，直接从语料库特征谱推导自适应γ(k)，无需标签数据或验证搜索。大量实验表明，SpecTemp在各种设置下均接近网格搜索最优γ* (k)的性能，误差低于1%，且完全无需学习、模型无关。代码已开源：https://github.com/liyongkang123/SpecTemp。
原文摘要 · Abstract (English)
Dimensionality reduction is critical for deploying dense retrieval systems at scale, yet mainstream post-hoc methods face a fundamental trade-off: principal component analysis (PCA) preserves dominant variance but underutilizes representational capacity, while whitening enforces isotropy at the cost of amplifying noise in the heavy-tailed eigenspectrum of retrieval embeddings. Intermediate spectral scaling methods unify these extremes by reweighting dimensions with a power coefficient $γ$, but treat $γ$ as a fixed hyperparameter that requires task-specific tuning. We show that the optimal scaling strength $γ$ is not a global constant: it varies systematically with target dimensionality $k$ and is governed by the signal-to-noise ratio (SNR) of the retained subspace. Based on this insight, we propose Spectral Tempering (\textbf{SpecTemp}), a learning-free method that derives an adaptive $γ(k)$ directly from the corpus eigenspectrum using local SNR analysis and knee-point normalization, requiring no labeled data or validation-based search. Extensive experiments demonstrate that Spectral Tempering consistently achieves near-oracle performance relative to grid-searched $γ^*(k)$ while remaining fully learning-free and model-agnostic. Our code is publicly available at https://github.com/liyongkang123/SpecTemp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。