通过推理时随机优化，提升视频动作迁移的多样性与真实感。
Inference-time Stochastic Refinement of GRU-Normalizing Flow for Real-time Video Motion Transfer
- 推理阶段引入马尔可夫链蒙特卡洛，增强模型输出多样性。
- 在长时序预测下仍保持高精度，生成轨迹更贴近真实数据分布。
- 适合对动作多样性和实时性要求高的游戏与异常检测场景。
实时视频动作迁移应用如沉浸式游戏和基于视觉的异常检测，需要准确且多样的未来预测以支持真实合成与不确定条件下的鲁棒决策。为提升序列预测的多样性，我们提出一种新颖的推理时精炼技术，将门控循环单元-归一化流（GRU-NF）与随机采样方法结合。尽管GRU-NF通过在时间预测框架中集成归一化流能捕捉多模态分布，但其确定性变换结构限制了表达能力。受随机归一化流（SNF）启发，我们在GRU-NF推理过程中引入马尔可夫链蒙特卡洛（MCMC）步骤，使模型能在不重训练的前提下探索更丰富的输出空间，更好地逼近真实数据分布。我们在基于关键点的视频动作迁移流程中验证该方法，其中生成时序一致且感知多样化的未来轨迹对于真实样本和低带宽通信至关重要。实验表明，所提出的推理框架——门控循环单元-随机归一化流（GRU-SNF），在生成多样性上优于GRU-NF，且在更长预测时域下未牺牲准确性。通过推理时注入随机性，该方法更有效地捕捉多模态行为。结果凸显了将随机动力学与基于流的序列模型结合在生成时间序列预测中的潜力。代码已开源：https://github.com/Tasmiah1408028/Inference-Time-Stochastic-Refinement-Of-GRU-NF-For-Real-Time-Video-Motion-Transfer
原文摘要 · Abstract (English)
Real-time video motion transfer applications such as immersive gaming and vision-based anomaly detection require accurate yet diverse future predictions to support realistic synthesis and robust downstream decision making under uncertainty. To improve the diversity of such sequential forecasts we propose a novel inference-time refinement technique that combines Gated Recurrent Unit-Normalizing Flows (GRU-NF) with stochastic sampling methods. While GRU-NF can capture multimodal distributions through its integration of normalizing flows within a temporal forecasting framework, its deterministic transformation structure can limit expressivity. To address this, inspired by Stochastic Normalizing Flows (SNF), we introduce Markov Chain Monte Carlo (MCMC) steps during GRU-NF inference, enabling the model to explore a richer output space and better approximate the true data distribution without retraining. We validate our approach in a keypoint-based video motion transfer pipeline, where capturing temporally coherent and perceptually diverse future trajectories is essential for realistic samples and low bandwidth communication. Experiments show that our inference framework, Gated Recurrent Unit- Stochastic Normalizing Flows (GRU-SNF) outperforms GRU-NF in generating diverse outputs without sacrificing accuracy, even under longer prediction horizons. By injecting stochasticity during inference, our approach captures multimodal behavior more effectively. These results highlight the potential of integrating stochastic dynamics with flow-based sequence models for generative time series forecasting. The code is available at: https://github.com/Tasmiah1408028/Inference-Time-Stochastic-Refinement-Of-GRU-NF-For-Real-Time-Video-Motion-Transfer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。