用图结构增强大模型，直接从语音中识别对话意图。
DialogGraph-LLM: Graph-Informed LLMs for End-to-End Audio Dialogue Intent Recognition
- 构建多关系对话注意力网络，融合语音与上下文图信息
- 在真实对话数据上达到92.3%准确率，优于现有基线
- 适合语音客服、会议记录等少标注场景
长时语音对话中的说话人意图识别具有广泛应用价值，但因话语间复杂依赖关系和标注数据稀缺而极具挑战。本文提出端到端框架DialogGraph-LLM，结合新型多关系对话注意力网络（MR-DAN）与多模态基础模型（如Qwen2.5-Omni-7B），实现从声学信号直接推断意图。设计自适应半监督学习策略，基于全局与类别置信度双重阈值过滤生成伪标签，并通过熵选择高信息量未标注样本。在自有MarketCalls语料库与公开MIntRec 2.0基准上评估表明，该框架显著优于强音频与文本驱动基线，在真实场景下展现优异性能与效率，证明其在弱监督音频密集领域中的实用价值。代码已开源。
原文摘要 · Abstract (English)
Recognizing speaker intent in long audio dialogues among speakers has a wide range of applications, but is a non-trivial AI task due to complex inter-dependencies in speaker utterances and scarce annotated data. To address these challenges, an end-to-end framework, namely DialogGraph-LLM, is proposed in the current work. DialogGraph-LLM combines a novel Multi-Relational Dialogue Attention Network (MR-DAN) architecture with multimodal foundation models (e.g., Qwen2.5-Omni-7B) for direct acoustic-to-intent inference. An adaptive semi-supervised learning strategy is designed using LLM with a confidence-aware pseudo-label generation mechanism based on dual-threshold filtering using both global and class confidences, and an entropy-based sample selection process that prioritizes high-information unlabeled instances. Extensive evaluations on the proprietary MarketCalls corpus and the publicly available MIntRec 2.0 benchmark demonstrate DialogGraph-LLM's superiority over strong audio and text-driven baselines. The framework demonstrates strong performance and efficiency in intent recognition in real world scenario audio dialogues, proving its practical value for audio-rich domains with limited supervision. Our code is available at https://github.com/david188888/DialogGraph-LLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。