用语音节奏谱分析痴呆，提升分类准确率
Leveraging AM and FM Rhythm Spectrograms for Dementia Classification and Assessment
- 提出节奏形式分析谱作为新特征，捕捉长期语音时序模式
- 手工特征比eGeMAPs高14.2%分类准确率，回归表现相当
- 融合视觉变压器与语言嵌入，优于梅尔谱，适合临床评估
本研究探索节奏形式分析（RFA）在捕捉痴呆患者语音中长期时间调制方面的潜力。我们引入由RFA生成的节奏谱作为痴呆分类与回归任务的新特征。提出两种方法：(1) 基于节奏谱的手工特征；(2) 数据驱动的融合方法，将RFA节奏谱与视觉变换器（ViT）生成的声学表示及BERT语言嵌入结合。与现有特征对比，结果表明：手工特征在分类任务中相对提升14.2%准确率，回归表现与基线相当；融合方法中，RFA谱在分类上较梅尔谱相对提升约13.1%，回归性能与基线相当。
原文摘要 · Abstract (English)
This study explores the potential of Rhythm Formant Analysis (RFA) to capture long-term temporal modulations in dementia speech. Specifically, we introduce RFA-derived rhythm spectrograms as novel features for dementia classification and regression tasks. We propose two methodologies: (1) handcrafted features derived from rhythm spectrograms, and (2) a data-driven fusion approach, integrating proposed RFA-derived rhythm spectrograms with vision transformer (ViT) for acoustic representations along with BERT-based linguistic embeddings. We compare these with existing features. Notably, our handcrafted features outperform eGeMAPs with a relative improvement of $14.2\%$ in classification accuracy and comparable performance in the regression task. The fusion approach also shows improvement, with RFA spectrograms surpassing Mel spectrograms in classification by around a relative improvement of $13.1\%$ and a comparable regression score with the baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。