对比了语音识别中两种主流模型的处理策略差异。
Categorize Early, Integrate Late: Divergent Processing Strategies in Automatic Speech Recognition
- 通过分离架构影响，发现卷积注意力模型更早分类音素和说话人性别。
- 变压器模型将音素、口音等编码推迟到深层，延迟49%~57%。
- 适合低延迟场景或需上下文建模的任务，可指导模型设计。
在语音语言建模中，两大主流架构——Transformer 和 Conformer——性能相当，但其成功是否源于相似的处理方式，还是受不同架构先验偏见驱动尚不明确。本文提出‘架构指纹’探测框架，隔离架构对表征的影响，并应用于24个预训练编码器（参数量39M-3.3B）的受控实验。分析揭示二者存在显著分异：Conformer采用‘早期分类’策略，在深度上提前29%完成音素类别判断，16%深度分辨说话人性别；而Transformer则采取‘晚期整合’，将音素、口音和持续时间编码延至深层（49%-57%层）。这些特征为设计提供启发：Conformer的前端分类利于低延迟流式处理，而Transformer的深层整合更适合需要丰富上下文与跨话语归一化的任务。
原文摘要 · Abstract (English)
In speech language modeling, two architectures dominate the frontier: the Transformer and the Conformer. However, it remains unknown whether their comparable performance stems from convergent processing strategies or distinct architectural inductive biases. We introduce Architectural Fingerprinting, a probing framework that isolates the effect of architecture on representation, and apply it to a controlled suite of 24 pre-trained encoders (39M-3.3B parameters). Our analysis reveals divergent hierarchies: Conformers implement a "Categorize Early" strategy, resolving phoneme categories 29% earlier in depth and speaker gender by 16% depth. In contrast, Transformers "Integrate Late," deferring phoneme, accent, and duration encoding to deep layers (49-57%). These fingerprints suggest design heuristics: Conformers' front-loaded categorization may benefit low-latency streaming, while Transformers' deep integration may favor tasks requiring rich context and cross-utterance normalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。