用动态参数记忆让大模型记住长对话情绪，突破上下文长度限制。
Dynamic Parameter Memory: Temporary LoRA-Enhanced LLM for Long-Sequence Emotion Recognition in Conversation
- 引入临时LoRA模块动态存储对话情绪上下文信息
- 在IEMOCAP上实现长音频下最优情绪识别性能
- 适合需要处理长时间对话情绪的任务场景
近期研究将语音大语言模型（SLLM）应用于语音情绪识别（SER），但语音模态固有的高帧率严重限制了SLLM的信号处理与理解能力。例如，拥有4K上下文窗口的SLLM在50Hz特征采样率下，仅能处理80秒音频即达容量上限。现有输入令牌压缩方法忽略了情绪在多轮对话中的连续性与惯性。本文提出动态参数记忆（DPM）机制，结合上下文语义与句级情绪编码，使SLLM在有限上下文窗口下也能处理无限长度音频。具体而言，DPM在推理过程中逐步将句级信息与情绪编码至临时LoRA模块，实现有效“记忆”。我们以情绪化SLLM为骨干，在推理阶段集成DPM用于对话情绪识别（ERC）。在IEMOCAP数据集上的实验表明，DPM显著提升了SLLM在长音频序列下的情绪识别能力，达到当前最优性能。
原文摘要 · Abstract (English)
Recent research has focused on applying speech large language model (SLLM) to improve speech emotion recognition (SER). However, the inherently high frame rate in speech modality severely limits the signal processing and understanding capabilities of SLLM. For example, a SLLM with a 4K context window can only process 80 seconds of audio at 50Hz feature sampling rate before reaching its capacity limit. Input token compression methods used in SLLM overlook the continuity and inertia of emotions across multiple conversation turns. This paper proposes a Dynamic Parameter Memory (DPM) mechanism with contextual semantics and sentence-level emotion encoding, enabling processing of unlimited-length audio with limited context windows in SLLM. Specifically, DPM progressively encodes sentence-level information and emotions into a temporary LoRA module during inference to effectively "memorize" the contextual information. We trained an emotion SLLM as a backbone and incorporated our DPM into inference for emotion recognition in conversation (ERC). Experimental results on the IEMOCAP dataset show that DPM significantly improves the emotion recognition capabilities of SLLM when processing long audio sequences, achieving state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。