通过动态路由与合并机制,实现高效视频超分辨率重建。
TRaM-VSR: Importance-Aware Token Routing and Merging for One-Step Diffusion Video Super-Resolution

- 根据运动与语义信息评估帧内关键令牌,自适应分配处理资源。
- 推理速度提升显著,同时保持顶尖重建质量与时间一致性。
- 适合追求高效高质视频重建的开发者和研究者使用。
基于大规模扩散变换器(DiT)先验的视频超分辨率(VSR)虽能实现极佳感知质量,但因密集时空令牌序列处理带来二次方计算开销,难以实用。现有高效方法易导致不可逆细节丢失与时间闪烁,尤其在单步扩散模型中更为明显。为此,本文提出TRaM-VSR,一种面向自适应令牌分配的令牌路由与合并框架,融合上下文感知视频先验与网络级先验。首先,通过融合运动敏感的时间线索与语义文本相似性,估算令牌重要性,识别动态物体与结构边界;其次,利用离线规划器校准重要性,指导跨最优分组网络块的路由。技术上,每个路由组内,结构关键令牌在高保真局部流中处理,非关键令牌则聚合为紧凑全局流,二者均受网络深度调制,并契合扩散模型的多粒度特性。大量实验表明,TRaM-VSR显著加速推理,同时保持当前最佳重建质量与鲁棒时间一致性。代码已公开于https://github.com/Ree1s/TRaM-VSR。
原文摘要 · Abstract (English)
Video super-resolution (VSR) using large-scale Diffusion Transformer (DiT) priors achieves exceptional perceptual quality but is often impractical due to the quadratic computational cost of processing dense spatio-temporal token sequences. Existing efficiency-oriented methods risk irreversible detail loss and temporal flickering, a vulnerability especially pronounced in one-step diffusion models. To address this, we propose TRaM-VSR, a Token Routing and Merging framework for adaptive token allocation, leveraging both context-aware video priors and network-level priors. First, token importance is estimated by fusing motion-sensitive temporal cues with semantic text similarity, isolating dynamic objects and structural boundaries. Next, this importance is further calibrated and adjusted by an offline planner to guide routing across optimally grouped network blocks. Technically, within each routed group, structurally critical tokens are processed in a high-fidelity local stream, while less informative tokens are aggregated into a compact global stream, both modulated by network depth and aligned with the multigranular nature of diffusion models. Extensive experiments show that TRaM-VSR accelerates inference significantly while preserving state-of-the-art reconstruction quality and robust temporal consistency. The code is available at https://github.com/Ree1s/TRaM-VSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。