arXiv:2601.01745cs.CLcs.AI2026-01AAAI

提出双向交互注意力框架,提升多粒度发音评估精度

Multi-granularity Interactive Attention Framework for Residual Hierarchical Pronunciation Assessment

  • 设计双向注意力模块,打通音素、词、语句层级的互动
  • 在speechocean762数据集上超越现有最佳方法
  • 适合语音教学系统与自动发音评测研发者

自动发音评估在计算机辅助发音训练系统中至关重要。由于能同时处理多种发音任务,多方面多粒度评估方法逐渐受到关注,并在性能上优于单层次建模。然而,现有方法仅考虑相邻粒度层间的单向依赖关系,缺乏音素、词和语句层级间的双向交互,因而未能充分捕捉声学结构关联。为此,我们提出一种新颖的残差层级交互方法(简称HIA),实现跨粒度的双向建模。HIA的核心是交互注意力模块,利用注意力机制实现动态双向交互,有效捕捉各粒度的语言特征并融合不同层级间的相关性。同时,我们引入残差层级结构以缓解建模声学层级时的特征遗忘问题。此外,使用一维卷积层增强各粒度下的局部上下文线索提取。在speechocean762数据集上的大量实验表明,该模型全面优于现有最先进方法。

原文摘要 · Abstract (English)

Automatic pronunciation assessment plays a crucial role in computer-assisted pronunciation training systems. Due to the ability to perform multiple pronunciation tasks simultaneously, multi-aspect multi-granularity pronunciation assessment methods are gradually receiving more attention and achieving better performance than single-level modeling tasks. However, existing methods only consider unidirectional dependencies between adjacent granularity levels, lacking bidirectional interaction among phoneme, word, and utterance levels and thus insufficiently capturing the acoustic structural correlations. To address this issue, we propose a novel residual hierarchical interactive method, HIA for short, that enables bidirectional modeling across granularities. As the core of HIA, the Interactive Attention Module leverages an attention mechanism to achieve dynamic bidirectional interaction, effectively capturing linguistic features at each granularity while integrating correlations between different granularity levels. We also propose a residual hierarchical structure to alleviate the feature forgetting problem when modeling acoustic hierarchies. In addition, we use 1-D convolutional layers to enhance the extraction of local contextual cues at each granularity. Extensive experiments on the speechocean762 dataset show that our model is comprehensively ahead of the existing state-of-the-art methods.

发音评估注意力机制多粒度语音分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。