提出可扩展的视频翻译架构,解决多用户会议实时性难题
Generative AI for Video Translation: A Scalable Architecture for Multilingual Video Conferencing
- 用轮换机制将计算复杂度从平方级降至线性
- 在多种硬件上实现延迟低于1秒的实时处理
- 适合构建多语言视频会议等实时生成应用
实时部署视频翻译等级联生成AI流水线面临系统级挑战,包括顺序模型推理的累积延迟和多用户场景下$$\mathcal{O}(N^2)$的计算复杂度,导致系统不可扩展。本文提出并评估了一种实用的系统级框架,以缓解这些关键瓶颈。该架构引入轮换机制,将多用户场景下的计算复杂度从二次方降至线性;同时采用分段处理协议,控制推理延迟以实现感知上的实时体验。我们实现了一个概念验证流水线,并在包含消费级(NVIDIA RTX 4060)、云服务(NVIDIA T4)和企业级(NVIDIA A100)GPU的多层次硬件环境下进行了严格的性能分析。客观评估表明,系统在现代硬件上实现了实时吞吐量(τ<1.0)。主观用户研究进一步验证了该方法:用户高度接受初始可预测的处理延迟,以换取流畅无中断的播放体验。本工作提供了经验证的端到端系统设计,为多语言通信平台中可扩展的实时生成AI应用部署提供了可行路径。
原文摘要 · Abstract (English)
The real-time deployment of cascaded generative AI pipelines for applications like video translation is constrained by significant system-level challenges. These include the cumulative latency of sequential model inference and the quadratic ($\mathcal{O}(N^2)$) computational complexity that renders multi-user video conferencing applications unscalable. This paper proposes and evaluates a practical system-level framework designed to mitigate these critical bottlenecks. The proposed architecture incorporates a turn-taking mechanism to reduce computational complexity from quadratic to linear in multi-user scenarios, and a segmented processing protocol to manage inference latency for a perceptually real-time experience. We implement a proof-of-concept pipeline and conduct a rigorous performance analysis across a multi-tiered hardware setup, including commodity (NVIDIA RTX 4060), cloud (NVIDIA T4), and enterprise (NVIDIA A100) GPUs. Our objective evaluation demonstrates that the system achieves real-time throughput ($τ< 1.0$) on modern hardware. A subjective user study further validates the approach, showing that a predictable, initial processing delay is highly acceptable to users in exchange for a smooth, uninterrupted playback experience. The work presents a validated, end-to-end system design that offers a practical roadmap for deploying scalable, real-time generative AI applications in multilingual communication platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。