让游戏解说实时生成，消除说话间隙的延迟
Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation
- 文本生成与语音播放并行，提前缓冲多个解说候选
- 解说间隔静默从9.6秒降至0.3秒，提升40%自然度
- 适合需要低延迟实时解说的电竞直播、游戏应用
我们提出一种低延迟的实时音频游戏解说系统，可直接从实时游戏画面生成语音解说。在端到端设置中，主要瓶颈是累积等待时间：传统流水线对每个解说片段依次完成帧捕获、文本生成和语音合成，且必须等语音播放结束后才开始下一次生成，这种严格串行导致解说间出现长时间不自然的沉默。为解决此延迟瓶颈，我们的系统在语音播放的同时并行进行文本生成，并提前缓存多个候选解说内容，实现在播放边界即时合成。在快节奏游戏视频上的实验表明，相比串行基线，本方法将平均解说间隔静默时间从9.6秒降低至0.3秒；同时，解说节奏与专业解说的静默模式相似度提升超过40%。120名经验丰富的玩家参与的用户研究证实，系统显著提升了语音节奏的感知自然度。演示视频见：https://youtu.be/pmrRUlvav8M。
原文摘要 · Abstract (English)
We present a low-latency real-time audio game commentary system that generates spoken commentary directly from live gameplay video. In this end-to-end setting, a key bottleneck is accumulated waiting time; conventional pipelines capture frames, generate text, and synthesize speech sequentially for each utterance, and do not request the next generation until speech playback has completed. This strict sequentiality causes long and unnatural silence between utterances. To address this latency bottleneck, our system runs text generation in parallel with speech playback and buffers multiple candidate utterances ahead of time, enabling immediate synthesis at playback boundaries. Experiments on fast-paced game videos show that our parallel design reduces the mean inter-utterance silence from 9.6 seconds to 0.3 seconds compared to sequential baselines. It also improves similarity to professional speaking--silence timing patterns by over 40 %, and a user study with 120 experienced game players confirms significantly improved perceived speaking rhythm. Our demo video is available at: https://youtu.be/pmrRUlvav8M.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。