用混合模型实现视频实时稳定，减少抖动和画面裁切。
TranStable: Towards Robust Pixel-level Online Video Stabilization by Jointing Transformer and CNN
- 结合Transformer与CNN的TUNet生成全局与局部特征融合的像素级变形图。
- 在NUS、DeepStab等数据集上达到最优性能，有效减少抖动并保留更多画面。
- 适合需要高质量视频稳定的应用，如无人机拍摄与自拍视频处理。
视频稳定常面临失真和过度裁切问题。本文提出端到端框架TranStable，包含生成器与判别器。生成器采用TransformerUNet（TUNet），通过分层自适应融合模块（HAFM）整合Transformer与CNN，利用多视觉线索的全局与局部特征，建模帧间关系，生成鲁棒的像素级形变图以实现几何稳定。此外，设计稳定性判别模块（SDM），在训练中提供像素级监督，确保真实性和一致性，提升画面完整性，减少抖动伪影，增强视觉保真度。在NUS、DeepStab和Selfie基准测试中表现优异，达到当前最佳水平。
原文摘要 · Abstract (English)
Video stabilization often struggles with distortion and excessive cropping. This paper proposes a novel end-to-end framework, named TranStable, to address these challenges, comprising a genera tor and a discriminator. We establish TransformerUNet (TUNet) as the generator to utilize the Hierarchical Adaptive Fusion Module (HAFM), integrating Transformer and CNN to leverage both global and local features across multiple visual cues. By modeling frame-wise relationships, it generates robust pixel-level warping maps for stable geometric transformations. Furthermore, we design the Stability Discriminator Module (SDM), which provides pixel-wise supervision for authenticity and consistency in training period, ensuring more complete field-of-view while minimizing jitter artifacts and enhancing visual fidelity. Extensive experiments on NUS, DeepStab, and Selfie benchmarks demonstrate state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。