实时预测高清无人机视频，突破速度与画质的矛盾
RAPTOR: Real-Time High-Resolution UAV Video Prediction with Efficient Video Attention
- 单次前向传播+分治注意力机制，大幅降低计算开销
- 512²分辨率下仍达30帧以上，首次在边缘设备实现
- 适合对延迟敏感的无人机导航等真实场景
视频预测长期面临高分辨率、高感知质量与实时性难以兼得的困境，尤其在密集城市环境中，自主无人机需基于高清影像预判事件以保障安全。现有方法依赖迭代生成或二次复杂度注意力，难以满足边缘硬件的严苛要求。为此，本文提出RAPTOR，一种实现实时高分辨率视频预测的新架构。其单次前向设计避免了迭代过程中的误差累积与延迟；核心创新为高效视频注意力（EVA），通过交替处理空间与时间维度，将时空建模复杂度从O((ST)²)降至O(S+T)，内存需求降至O(max(S,T))，可在512²及以上分辨率上直接处理密集特征图，无需分块。配合三阶段训练策略，逐步优化从粗略结构到精细时序一致细节的预测。实验表明，RAPTOR是首个在Jetson AGX Orin上实现512²分辨率超30 FPS的预测模型，在UAVid、KTH及自建高分辨率数据集上各项指标领先。更重要的是，其在真实无人机导航任务中使任务成功率提升18%，推动更安全、更具预见性的智能体发展。
原文摘要 · Abstract (English)
Video prediction is plagued by a fundamental trilemma: achieving high-resolution and perceptual quality typically comes at the cost of real-time speed, hindering its use in latency-critical applications. This challenge is most acute for autonomous UAVs in dense urban environments, where foreseeing events from high-resolution imagery is non-negotiable for safety. Existing methods, reliant on iterative generation (diffusion, autoregressive models) or quadratic-complexity attention, fail to meet these stringent demands on edge hardware. To break this long-standing trade-off, we introduce RAPTOR, a video prediction architecture that achieves real-time, high-resolution performance. RAPTOR's single-pass design avoids the error accumulation and latency of iterative approaches. Its core innovation is Efficient Video Attention (EVA), a novel translator module that factorizes spatiotemporal modeling. Instead of processing flattened spacetime tokens with $O((ST)^2)$ or $O(ST)$ complexity, EVA alternates operations along the spatial (S) and temporal (T) axes. This factorization reduces the time complexity to $O(S + T)$ and memory complexity to $O(max(S, T))$, enabling global context modeling at $512^2$ resolution and beyond, operating directly on dense feature maps with a patch-free design. Complementing this architecture is a 3-stage training curriculum that progressively refines predictions from coarse structure to sharp, temporally coherent details. Experiments show RAPTOR is the first predictor to exceed 30 FPS on a Jetson AGX Orin for $512^2$ video, setting a new state-of-the-art on UAVid, KTH, and a custom high-resolution dataset in PSNR, SSIM, and LPIPS. Critically, RAPTOR boosts the mission success rate in a real-world UAV navigation task by 18%, paving the way for safer and more anticipatory embodied agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。