用语音活动检测边界实现上下文感知的交错批处理，提升长音频转录准确率。
Context-Aware Interleaved Batching for WhisperX
- 基于语音活动检测划分音频段，交错批处理时保持连续上下文
- 在长音频上降低词错误率，改善专有名词识别
- 兼顾高吞吐量推理与上下文连贯性，适合实时长语音应用
WhisperX 通过音频内批处理加速语音转录，但会隔离音频片段，丢失连贯标点和术语转录所需的历史上下文。标准 Whisper 虽能保留顺序上下文，却存在推理慢和幻觉循环问题。为此，我们提出上下文感知的交错批处理方法。利用语音活动检测（VAD）生成的分段边界，算法稳定了 Whisper 的文本条件，使我们可在批处理的音频片段间安全维持连续历史上下文。在长音频基准测试中，该方法显著降低词错误率（WER），提升专有名词转录效果，同时保持高吞吐量推理速度。
原文摘要 · Abstract (English)
While WhisperX accelerates speech transcription via intra-audio batching, it isolates audio segments, losing the historical context needed for coherent punctuation and terminology transcription. Conversely, standard Whisper retains context sequentially but suffers from slow inference and hallucination loops. To achieve the best of both worlds, we propose Context-Aware Interleaved Batching. By using VAD-derived segment boundaries, our algorithm stabilizes Whisper's text conditioning, allowing us to safely maintain continuous historical context across batched audio segments. As demonstrated on long-form audio benchmarks, this approach reduces Word Error Rate (WER) and improves proper noun transcription, all while maintaining high-throughput inference speeds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。