arXiv:2507.12951eess.AScs.AI2025-07被引 1

统一建模语音理解多任务,提升跨任务协同与系统效率

UniSLU: Unified Spoken Language Understanding from Heterogeneous Cross-Task Datasets

  • 用统一表示融合不同语音任务数据,支持跨任务共享
  • 联合建模语音识别、实体识别和情感分析,性能优于基线方法
  • 兼容大模型生成能力,适合真实语音场景应用

语音语言理解(SLU)在会议、访谈、客服等以语音为中心的多媒体应用中至关重要,涵盖自动语音识别(ASR)、语音命名实体识别(NER)和语音情感分析(SA)等任务。现有方法通常为各任务分别设计模型,导致系统复杂、缺乏跨任务交互,且未能充分利用多任务异构数据。为此,我们提出UniSLU,一个统一框架,在单一架构内联合建模多个SLU任务。通过构建统一表示,充分挖掘多任务异构数据;在此基础上,提出统一生成方法,联合建模ASR、语音NER和语音SA,增强任务间协作,并可无缝集成大语言模型,发挥其强大生成能力。在多个公开SLU数据集上的实验表明,该方法显著优于多个基准模型,适用于实际语音多媒体场景。代码与模型将开源至GitHub,促进后续研究。

原文摘要 · Abstract (English)

Spoken Language Understanding (SLU) plays a crucial role in speech-centric multimedia applications, enabling machines to comprehend spoken language in scenarios such as meetings, interviews, and customer service interactions. SLU encompasses multiple tasks, including Automatic Speech Recognition (ASR), spoken Named Entity Recognition (NER), and spoken Sentiment Analysis (SA). However, existing methods often rely on separate model architectures for individual tasks such as spoken NER and SA, which increases system complexity, limits cross-task interaction, and fails to fully exploit heterogeneous datasets available across tasks. To address these limitations, we propose UniSLU, a unified framework that jointly models multiple SLU tasks within a single architecture. Specifically, we propose a unified representation for diverse SLU tasks, enabling full utilization of heterogeneous datasets across multiple tasks. Built upon this representation, we propose a unified generative method that jointly models ASR, spoken NER, and SA tasks, enhancing task interactions and enabling seamless integration with large language models to harness their powerful generative capabilities. Extensive experiments on public SLU datasets demonstrate the effectiveness of our approach, achieving superior SLU performance compared to several benchmark methods, making it well-suited for real-world speech-based multimedia scenarios. We will release all code and models at github to facilitate future research.

语音理解多任务学习统一建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。