单目图像上高效实现多任务空间感知，提升精度与实时性
M2H: Multi-Task Learning with Efficient Window-Based Cross-Task Attention for Monocular Spatial Perception
- 用窗口化跨任务注意力机制，共享特征又保留任务特异性
- 在NYUDv2、Hypersim、Cityscapes上均优于现有方法，计算开销低
- 适合部署于边缘设备的实时3D场景理解系统，实测效果佳
在边缘设备上实现实时空间感知需要高效的多任务模型，以利用任务间互补信息同时降低计算开销。本文提出Multi-Mono-Hydra（M2H），一种针对单目图像的语义分割、深度、边缘和表面法线估计的新型多任务学习框架。不同于传统独立单任务模型或共享编码器-解码器架构，M2H引入基于窗口的跨任务注意力模块，实现结构化特征交互的同时保持任务特异性，提升多任务预测一致性。基于轻量级ViT-based DINOv2骨干网络，M2H专为实时部署优化，可支持动态环境中3D场景图构建。全面评估表明，M2H在NYUDv2上超越当前最优多任务模型，在Hypersim上超过单任务深度与语义基线，在Cityscapes上表现更优，且在笔记本硬件上保持高效。此外，其在真实数据上的验证也证明了实际应用价值。
原文摘要 · Abstract (English)
Deploying real-time spatial perception on edge devices requires efficient multi-task models that leverage complementary task information while minimizing computational overhead. This paper introduces Multi-Mono-Hydra (M2H), a novel multi-task learning framework designed for semantic segmentation and depth, edge, and surface normal estimation from a single monocular image. Unlike conventional approaches that rely on independent single-task models or shared encoder-decoder architectures, M2H introduces a Window-Based Cross-Task Attention Module that enables structured feature exchange while preserving task-specific details, improving prediction consistency across tasks. Built on a lightweight ViT-based DINOv2 backbone, M2H is optimized for real-time deployment and serves as the foundation for monocular spatial perception systems supporting 3D scene graph construction in dynamic environments. Comprehensive evaluations show that M2H outperforms state-of-the-art multi-task models on NYUDv2, surpasses single-task depth and semantic baselines on Hypersim, and achieves superior performance on the Cityscapes dataset, all while maintaining computational efficiency on laptop hardware. Beyond benchmarks, M2H is validated on real-world data, demonstrating its practicality in spatial perception tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。