让语音模型更聪明:用压缩推理提升数学问答准确率
Efficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Models

- 将文字推理压缩为语音引导的紧凑表示,降低计算开销
- 在数学问答上比标准方法高21%,仅用40%文本令牌
- 适合追求高效语音智能的开发者和研究者
语音语言模型(SLMs)虽能实现自然人机交互,但在数学问答任务上的推理能力仍落后于文本大模型。主要原因是其依赖难以解析的纯口语化表达。直接迁移文本推理框架受架构限制且计算成本高。为此,我们提出首个引入压缩推理的框架ECoM Reasoning,通过压缩文本成分,使其同时充当语音引导与推理表示,既提升准确率又减少令牌使用。训练时采用渐进式压缩策略,逐步从完整推理过渡到压缩推理。在多个语音数学问答基准测试中,ECoM Reasoning相比无显式推理的标准CoM提升21%准确率,相比带完整推理轨迹的CoM提升3%,同时仅消耗40%的文本令牌,显著增强推理能力且保持推理效率。
原文摘要 · Abstract (English)
Spoken language models (SLMs) enable natural human-computer interaction, but their reasoning ability still lags behind that of text-based large language models, especially on spoken mathematical question answering tasks. One important reason is that SLMs reason over purely verbalized mathematical expressions, which are harder to interpret than symbolic text. However, directly transferring text-based reasoning to SLMs is nontrivial due to architectural constraints and the additional computational requirements. To address this challenge, we propose Efficient Chain-of-Modality Reasoning (ECoM Reasoning), the first framework to introduce compressed reasoning into SLMs. By compressing the textual component so that it jointly serves as speech guidance and reasoning representation, ECoM Reasoning improves reasoning accuracy while using a smaller token budget than the standard Chain-of-Modality (CoM) architecture, which generates intermediate text before speech. To train this capability, we further propose Progressive Compression, a curriculum-based strategy that gradually trains the model from full-form reasoning to compressed reasoning. Experiments on spoken mathematical question answering benchmarks show that ECoM Reasoning improves accuracy by 21% over standard CoM without explicit reasoning, and by 3% over CoM with full reasoning traces while using only 40% of the text tokens, demonstrating that it enhances SLM reasoning while remaining inference-efficient.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。