用轻量卷积模块增强语音模型，提升跨域识别准确率。
GC-LoRA: Gated Convolutional LoRA for Parameter-Efficient Acoustic Adaptation
- 在Transformer中引入类Conformer的局部卷积结构
- 在多个噪声/方言数据集上降低10.9%错误率
- 仅新增少量参数，适合资源受限场景
基于Transformer的语音基础模型在多数自动语音识别任务中表现优异，但在声学特性不匹配的领域常出现性能下降。现有参数高效微调方法（如LoRA）虽能调整全局注意力，却缺乏对局部上下文建模的能力，难以捕捉领域特异性变化。本文提出GC-LoRA，一种新型适配器架构，在预训练Transformer编码器中注入类Conformer的局部卷积处理机制。通过在注意力输出投影层集成轻量适配器，该方法能有效捕捉局部声学依赖关系，同时不破坏预训练的全局表示。在多种数据集（声学退化、带宽受限、方言、儿童语音）上的实验表明，本方法相比基线可实现最高达10.9%的词错误率（WER）降低，且仅增加极少可训练参数。
原文摘要 · Abstract (English)
Transformer-based Speech Foundation Models excel in most Automatic Speech Recognition tasks but often suffer performance degradation when applied to domains with mismatched acoustic characteristics. While Parameter Efficient Fine-Tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA), adjust global attention, they lack the local context modeling crucial for capturing domain-specific variations. We propose GC-LoRA, a novel adapter architecture that injects Conformer-style local convolutional processing into pretrained Transformer encoders. By integrating a lightweight adapter to encoder attention output projections, our method efficiently captures local acoustic dependencies without disrupting pretrained global representations. Experiments across diverse datasets (acoustically-degraded, bandlimited, dialectal, child) demonstrate the efficacy of our approach, achieving Word Error Rate (WER) reductions of up to 10.9% compared to baselines while adding minimal trainable parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。