让语音识别同时识别说话人角色，提升医疗对话分析精度。
ASR-Synchronized Speaker-Role Diarization
- 并行训练角色预测网络，用高级语音特征提升判断准确率。
- 在公开和私有数据集上，角色识别错误率降低6.2%和4.5%。
- 适合需要区分医生、患者等角色的医疗对话场景使用。
说话人角色归因（RD）如医生与患者、律师与客户，比传统说话人归因（仅标为说话人1、说话人2）更具实用性。现有端到端ASR+RD方法使用单个变换器串行预测词与角色（角色置于发言结束时），但导致语音识别性能下降。为此，我们改进近期联合ASR+SD框架，冻结语音识别变换器，平行训练辅助角色变换器，为每个语音识别输出的词分配角色。首先，我们证明了说话人归因（SD）与角色归因（RD）是本质不同的任务，对声学与语言信息的依赖不同。基于此，提出：(1) 任务特异性预测网络；(2) 使用更深层的语音识别编码器特征作为角色编码器输入。此外，将共享空白的RNNT损失替换为沿最优强制对齐路径的交叉熵损失，进一步提升性能并降低训练时的计算与内存开销。在公开及私有医生-患者对话数据集上的实验表明，该方法相对最佳基线，角色词归因错误率（R-WDER）分别降低6.2%和4.5%。
原文摘要 · Abstract (English)
Speaker-role diarization (RD), such as doctor vs. patient or lawyer vs. client, is practically often more useful than conventional speaker diarization (SD), which assigns only generic labels (speaker-1, speaker-2). The state-of-the-art end-to-end ASR+RD approach uses a single transducer that serializes word and role predictions (role at the end of a speaker's turn), but at the cost of degraded ASR performance. To address this, we adapt a recent joint ASR+SD framework to ASR+RD by freezing the ASR transducer and training an auxiliary RD transducer in parallel to assign a role to each ASR-predicted word. For this, we first show that SD and RD are fundamentally different tasks, exhibiting different dependencies on acoustic and linguistic information. Motivated by this, we propose (1) task-specific predictor networks and (2) using higher-layer ASR encoder features as input to the RD encoder. Additionally, we replace the blank-shared RNNT loss by cross-entropy loss along the 1-best forced-alignment path to further improve performance while reducing computational and memory requirements during RD training. Experiments on a public and a private dataset of doctor-patient conversations demonstrate that our method outperforms the best baseline with relative reductions of 6.2% and 4.5% in role-based word diarization error rate (R-WDER), respectively
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。