为增强现实眼镜设计的语音分离模型，提升嘈杂环境下的目标说话人识别效果。
BiTSE: Binaural Target Speaker Extraction in Noisy Multi-Talker Environments for AR Glass Arrays
- 利用方向和语音活动信息引导双耳信号分离。
- 在SPEAR数据集上显著提升信号保真度与听感质量。
- 适合开发智能眼镜等可穿戴设备的语音处理系统。
在嘈杂多说话人对话场景中，分离出目标语音信号是增强现实（AR）可穿戴麦克风阵列系统的关键需求。本文提出一种双耳目标说话人分离（BiTSE）框架，结合空间与时间线索，即目标说话人的到达方向（DoA）及对应的语音活动信息，指导分离过程。基于双耳信号降噪架构，模型引入三项关键改进：(i) 使用循环位置嵌入的方位感知注意力机制；(ii) 基于时间戳的掩码策略，利用说话人活动信息抑制非目标语音段；(iii) 创新的两阶段损失优化策略，先训练模型实现鲁棒降噪，再微调以提升听觉感知质量。在语音增强于增强现实（SPEAR）挑战数据集上的评估表明，所提出的BiTSE持续优于传统方法，显著提升信号保真度与感知质量。
原文摘要 · Abstract (English)
Isolating a desired speech signal in noisy multi-talker conversational scenarios is a key requirement for augmented reality (AR) wearable microphone array systems. In this work, a binaural target speaker extraction (TSE) framework, termed BiTSE, is proposed. It leverages both spatial and temporal cues, specifically the direction-of-arrival (DoA) of the target speaker and corresponding voice activity information, to guide the extraction process. Built upon a binaural signal denoising architecture, our model integrates three key enhancements: (i) a DoA-aware attention mechanism using cyclic positional embeddings, (ii) a timestamp-based masking strategy that utilizes speaker activity to suppress non-target segments, and (iii) a novel two-stage loss optimization strategy that first trains the model for robust denoising and then fine-tunes it to improve perceptual quality. Evaluations on the SPeech Enhancement for Augmented Reality (SPEAR) challenge dataset demonstrate that the proposed BiTSE consistently improves upon conventional approaches, leading to enhanced signal fidelity and perceptual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。