arXiv:2605.02950cs.LGcs.AI2026-05

用轻量级方法替代复杂编码器,提升语义检索效率

Kernel Affine Hull Machines as Compute-Efficient Encoders for Frozen Semantic Spaces

论文配图:Kernel Affine Hull Machines as Compute-Efficient Encoders for Frozen Semantic Spaces
图 1 · 摘自论文原文
  • 基于核仿射包络机构建无反向传播的查询编码器
  • 在奥地利法律检索任务中实现0.504的MRR@20,提速8.53倍
  • 适合需要高效在线编码的固定语义空间部署场景

基于Transformer的语义编码器在检索任务中表现优异,但许多部署中的瓶颈在于在线查询编码而非离线文档索引。本文研究在强教师表示空间和索引固定后,是否可用更轻量且解析明确的估计器替代重复的神经查询编码。将固定教师的词到语义映射建模为条件均值估计问题,目标语义向量被表示为由后验聚类概率加权的语义原型噪声混合。利用核仿射包络机(KAHM)几何,在显式识别的再生核希尔伯特空间(RKHS)中,从廉价词汇特征估计后验权重,语义原型通过归一化最小均方更新从噪声教师嵌入中优化。这带来无需反向传播的查询端编码器,并实现端到端误差分解为后验近似、有限样本/泛化与教师噪声项。在包含5,000个测试查询、84个候选法律条文和10,762个对齐检索单元的奥地利法律检索基准上进行验证,使用特定法律编码器进入冻结的Mixedbread嵌入空间。在评估匹配的学习适配器中,KAHM实现了最强的教师空间重建和所有截断点下的最佳排序敏感检索性能:k=20时,MRR@20=0.504,Hit@20=0.694,Top-1 Accuracy=0.411,相比报告的CPU环境下直接使用Transformer编码查询,单次查询时间减少8.53倍。结果支持KAHM作为监督固定表示部署场景下的高效编码器。

原文摘要 · Abstract (English)

Transformer-based semantic encoders are effective for retrieval, but in many deployments the recurring bottleneck is online query encoding rather than offline corpus indexing. This paper studies whether, once a strong teacher representation space and corpus index are fixed, repeated neural query encoding can be replaced by a substantially lighter and analytically explicit estimator. We formulate fixed-teacher lexical-to-semantic encoding as a conditional-mean estimation problem in which the target semantic vector is represented as a noisy mixture of semantic prototypes weighted by posterior cluster probabilities. Kernel Affine Hull Machine (KAHM) geometry is used to estimate these posterior weights from inexpensive lexical features in an explicitly identified RKHS hypothesis space, and the semantic prototypes are refined by normalized least-mean-squares updates from noisy teacher embeddings. This yields a backpropagation-free query-side encoder together with an end-to-end error decomposition into posterior-approximation, finite-sample/generalization, and teacher-noise terms. We instantiate the approach on a controlled Austrian-law retrieval benchmark with 5,000 test queries, 84 candidate laws, and 10,762 aligned retrieval units, using law-specific encoders into a frozen Mixedbread embedding space. Among evaluation-matched learned adapters, KAHM achieves the strongest teacher-space reconstruction and the best rank-sensitive retrieval performance at all evaluated cutoffs. At k=20, it obtains MRR@20 = 0.504, Hit@20 = 0.694, and Top-1 Accuracy = 0.411, while reducing online per-query time by 8.53 relative to direct transformer query encoding in the reported CPU setting. The results support KAHMs as compute-efficient encoders for supervised fixed-representation deployment regimes.

语义检索轻量编码计算效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。