让语音大模型边听边思考,提升推理速度与准确率。
Can Speech LLMs Think while Listening?
- 用思维链微调让语音模型在文本空间推理,平均提升2.4倍准确率。
- 提出'问题完整度'指标,提前启动推理,相同延迟下准确率提升4%。
- 结合偏好优化技术,在不损失准确率前提下降低70%响应延迟。
近期语音大语言模型(speech LLMs)实现了流畅的口语交互,但在复杂推理任务上仍表现不佳。本文研究了多流语音模型的思维链(CoT)微调效果,发现文本空间推理可使语音模型在一系列口语推理任务中平均准确率提升2.4倍。除准确率外,语音响应延迟是影响人机交互体验的关键因素。受人类‘边听边思考’行为启发,本文提出在用户语句未结束前即开始推理的方法,并引入基于熵的‘问题完整度’指标,用于判断最优推理启动时机。该方法相比启发式策略能更好控制准确率与延迟的权衡,在等延迟条件下于ARC-Easy数据集上实现4%的准确率提升。最后,利用拒绝采样生成偏好数据,通过直接偏好优化(DPO)进一步推进准确率-延迟帕累托前沿,实现70%的延迟降低而无准确率损失。
原文摘要 · Abstract (English)
Recent advances in speech large language models (speech LLMs) have enabled seamless spoken interactions, but these systems still struggle with complex reasoning tasks. Previously, chain-of-thought (CoT) prompting or fine-tuning has been to shown to significantly improve the reasoning abilities of text-based LLMs. In this work, we investigate the effect of CoT fine-tuning for multi-stream speech LLMs, demonstrating that reasoning in text space improves the accuracy of speech LLMs by 2.4x, on average, over a suite of spoken reasoning tasks. Beyond accuracy, the latency of the spoken response is a crucial factor for interacting with voice-based agents. Inspired by the human behavior of "thinking while listening," we propose methods to reduce the additional latency from reasoning by allowing the model to start reasoning before the user query has ended. To achieve this, we introduce an entropy-based metric, "question completeness," which acts as an indicator to guide the model on the optimal time to start reasoning. This method provides greater control over the accuracy-latency trade-off compared with heuristic-based approaches and, under equivalent latency conditions, yields a 4% accuracy gain on ARC-Easy. Finally, we use Direct Preference Optimization (DPO) on preference data created using rejection sampling to push the accuracy-latency pareto frontier further, resulting in a 70% reduction in latency without loss in accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。