用分层策略优化提升无界语音同步翻译质量与效率
Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech
- 通过分层奖励机制优化低质量对话数据上的模型
- 在1.5秒延迟下翻译质量提升7点COMET和1.25点MetricX
- 适合需要高效高质语音翻译的实时应用开发者
同步语音翻译（SST）在接收部分语音输入时即生成翻译。近期研究显示，大语言模型（LLMs）可显著提升SST质量，但带来高计算开销。此前方法将SST重构为多轮对话任务，实现LLM键值缓存完全复用并消除冗余特征重计算。然而，该方法依赖对话形式的监督微调（SFT）数据，此类人工标注极少，现有合成方法无法保证数据质量。本文提出分层策略优化（HPO）方法，对在不完美SFT数据上训练的模型进行后训练。引入分层奖励，平衡翻译质量与延迟目标。在英译中/德/日语任务上，1.5秒延迟下实现超过+7 COMET得分和+1.25 MetricX得分的提升。全面消融实验验证了不同质量奖励、分层奖励设计及分割策略的有效性。代码见：https://github.com/owaski/HPO
原文摘要 · Abstract (English)
Simultaneous speech translation (SST) generates translations while receiving partial speech input. Recent advances show that large language models (LLMs) can substantially improve SST quality, but at the cost of high computational overhead. To reduce this cost, prior work reformulates SST as a multi-turn dialogue task, enabling full reuse of the LLM's key-value (KV) cache and eliminating redundant feature recomputation. However, this approach relies on supervised fine-tuning (SFT) data in dialogue form, for which few human annotations exist, and existing synthesis methods cannot guarantee data quality. In this work, we propose a Hierarchical Policy Optimization (HPO) approach that post-train models trained on imperfect SFT data. We introduce a hierarchical reward that balances translation quality and latency objectives. Experiments on English to Chinese/German/Japanese demonstrate improvements of over +7 COMET score and +1.25 MetricX score at a latency of 1.5 seconds. Comprehensive ablation studies further validate the effectiveness of different quality rewards, hierarchical reward formulations, and segmentation strategies. Code can be found here https://github.com/owaski/HPO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。