用语法解析切分语音流，提升实时翻译质量。
SASST: Leveraging Syntax-Aware Chunking and LLMs for Simultaneous Speech Translation
- 基于语法结构和标点切分输入，保证语义完整
- 在CoVoST2上多语言翻译准确率显著提升
- 适合需要低延迟高精度的实时翻译场景
本文提出一种基于语法的切分策略，通过分析依存关系（如名词短语边界、动宾结构）和标点特征，将输入语音流分割为语义完整的单元，确保切分连贯性并减少语义碎片化。在此基础上，我们构建了SASST（语法感知的实时语音翻译）端到端框架，集成冻结的Whisper编码器与解码仅的LLM。统一架构动态输出翻译词元或<WAIT>符号，联合优化翻译时机与内容，目标侧重排机制缓解词序偏差。在CoVoST2多语言数据集En-{De, Zh, Ja}上的实验表明，该方法在多种语言下均实现显著翻译质量提升，验证了语法结构在LLM驱动的实时语音翻译系统中的有效性。
原文摘要 · Abstract (English)
This work proposes a grammar-based chunking strategy that segments input streams into semantically complete units by parsing dependency relations (e.g., noun phrase boundaries, verb-object structures) and punctuation features. The method ensures chunk coherence and minimizes semantic fragmentation. Building on this mechanism, we present SASST (Syntax-Aware Simultaneous Speech Translation), an end-to-end framework integrating frozen Whisper encoder and decoder-only LLM. The unified architecture dynamically outputs translation tokens or <WAIT> symbols to jointly optimize translation timing and content, with target-side reordering addressing word-order divergence. Experiments on CoVoST2 multilingual corpus En-{De, Zh, Ja} demonstrate significant translation quality improvements across languages and validate the effectiveness of syntactic structures in LLM-driven SimulST systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。