BACH用贝叶斯混合头提升多兴趣召回，避免注意力坍缩并可复用用户兴趣权重。
BACH: A Bayesian Admixture of Contrastive Heads for Multi-Interest Two-Tower Retrieval
- 将多兴趣召回建模为用户级软混合头，通过变分推断训练
- 在3个大规模数据集上优于硬路由和单向量基线，且性能随头数提升
- 支持预计算检索，适合需要高效服务的推荐系统
双塔检索模型将每个用户压缩为单一嵌入，难以覆盖多样化兴趣。多兴趣模型虽引入多个头，但硬路由训练导致头利用率低（路由坍缩），且无法提供每个用户各兴趣的重要性估计。本文提出BACH（贝叶斯对比头混合），将多兴趣双塔召回建模为用户级别的头混合，通过变分推断拟合。该软混合策略训练所有头，缓解坍缩问题；生成可复用于服务的用户级兴趣权重，并支持共享全局码本，实现预计算检索。在MovieLens-20M、Taobao和Netflix三个大规模基准上，BACH在每种头数下均优于硬路由多兴趣及单向量基线。进一步发现，以最佳头评分候选项（与实际服务一致）优于传统目标路由训练，且此时BACH性能仍可提升。
原文摘要 · Abstract (English)
Two-tower retrievers compress each user into a single embedding, limiting their ability to serve diverse interests. Multi-interest models give each user several heads scored by a maximum inner product, but their hard-routing training under-utilizes heads (routing collapse) and gives no per-user estimate of how much each interest matters for serving. We present \textbf{BACH} (\emph{Bayesian Admixture of Contrastive Heads}), which casts multi-interest two-tower retrieval as a per-user mixture over the heads, fit by variational inference. The soft mixture trains every head (mitigating collapse), produces a per-user weighting of the interests that is reused at serving, and admits a shared global-codebook variant with precomputable retrieval. On three large-scale benchmarks, MovieLens-20M, Taobao, and Netflix, BACH improves top-of-ranking retrieval over hard-routing multi-interest and single-vector baselines at every head count; we further find that scoring every candidate by its best head, consistent with serving, outperforms the usual target-routed training, and that BACH improves further still.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。