混合生成模型实现低延迟手语实时合成，兼顾质量与速度。
Hybrid Autoregressive-Diffusion Model for Real-Time Sign Language Production
- 结合自回归与扩散模型，先快速生成帧再精细优化。
- 在How2Sign数据集上首帧时间仅5.9秒，每秒处理10.17帧。
- 适合需要实时手语生成的无障碍应用或人机交互场景。
早期手语生成（SLP）模型多采用自回归解码，虽保持时序因果性但存在推理误差累积问题。近期基于扩散的方法通过迭代去噪提升生成质量，却引入显著延迟。为此，本文提出HybridSign：一种结合因果帧生成与流形扩散精修的混合模型，实现低延迟手语生成。该模型采用多尺度姿态表示模块捕捉细微动作特征，并引入置信度感知因果注意力机制，利用关节点置信度提升对噪声2D姿态观测的鲁棒性。在PHOENIX14T和How2Sign数据集上的实验表明，HybridSign在对比基线中始终取得最佳质量-效率平衡。在How2Sign测试集上，达到BLEU-1/4为30.12/6.48，DTW为3.89，同时将首帧时间降至5.90秒，吞吐量提升至10.17 FPS（60帧评估协议下）。
原文摘要 · Abstract (English)
Earlier Sign Language Production (SLP) models typically relied on autoregressive decoding, which naturally preserves temporal causality but suffers from error accumulation at inference time. More recent diffusion-based approaches improve generation quality through iterative denoising, yet their sequence-level refinement process introduces substantial latency. To address this trade-off, we propose HybridSign, a hybrid autoregressive-diffusion model for low-latency sign language production that combines causal frame generation with flow-based diffusion refinement. A Multi-Scale Pose Representation module captures fine-grained articulator features, while a Confidence-Aware Causal Attention mechanism leverages joint-level confidence scores to improve robustness under noisy 2D pose observations. Experiments on PHOENIX14T and How2Sign show that HybridSign consistently achieves the best quality--efficiency trade-off among the compared baselines. On the How2Sign test split, it reaches BLEU-1/4 scores of 30.12/6.48 and DTW of 3.89, while reducing time-to-first-frame to 5.90s and increasing throughput to 10.17 FPS under a 60-frame evaluation protocol.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。