将语音属性分拆到向量不同维度，实现精准语义检索
Multi-Axis Speech Similarity via Factor-Partitioned Embeddings
- 用子空间分解语音的多维特征，如说话人、语义、口音等
- 通过加权余弦相似度，可抑制同说话人偏差并匹配跨条件语义
- 适合需要解耦语音属性的语音检索与跨域应用
语音同时携带语言内容、说话人身份、方言、性别等多种属性，传统单向量嵌入会将其混淆。本文提出因子分割嵌入框架，将每个语音片段映射为一个向量，其子空间对应不同变化轴。共享声学编码器连接各属性的线性投影头，分别通过专家教师蒸馏或共享标签对的对比学习训练。生成的嵌入支持属性条件检索：相似度为各轴余弦得分的带符号加权和，可联合考虑说了什么和怎么说，或显式抑制某一属性以凸显另一属性。在共享哈佛句子提示的跨语料库检索任务上评估，结果表明带符号轴加权能有效抑制同说话人偏差，提升跨录制条件的语义匹配能力。代码已公开：https://github.com/jimregan/spoken-sentence-transformers
原文摘要 · Abstract (English)
Speech encodes multiple simultaneous attributes -- linguistic content, speaker identity, dialect, gender --that conventional single-vector embeddings conflate. We present a factor-partitioned embedding framework that maps each utterance into a single vector whose subspaces correspond to distinct axes of variation. A shared acoustic encoder feeds per-axis linear projection heads, each trained via distillation from a specialist teacher or a contrastive objective over shared-label pairs. The resulting embeddings support attribute-conditioned retrieval: similarity is computed as a signed weighted sum over per-axis cosine scores, allowing retrieval that jointly considers what was said and how -- or explicitly suppresses one attribute to surface another. We evaluate on cross-corpus retrieval over corpora sharing the Harvard sentence prompts, demonstrating that signed axis weighting can suppress same-speaker bias and surface semantically matched utterances across recording conditions. Code is available at: https://github.com/jimregan/spoken-sentence-transformers
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。