arXiv:2607.21592cs.CV2026-07

一个模型同时预测8种视频场景属性,无需标注重叠或伪标签。

Unified Video Dense Prediction from Disjoint Data

论文配图:Unified Video Dense Prediction from Disjoint Data
图 1 · 摘自论文原文
  • 用任务专家蒸馏指导统一主干网络,避免标注重叠。
  • 在8个不同数据集上训练,仍保持强泛化能力。
  • 适合需要多任务统一建模的视觉理解研究者。

场景理解需要同时预测几何、外观和语义等信息,但现有任务标注分散在互不兼容的领域特定数据集上。当前统一系统要么仅限于完全共标注数据,要么需付出巨大计算成本进行伪标注。为此,我们提出UniD,一个统一视频模型,可联合预测八项密集场景属性:深度、表面法线、语义分割、边界、人体部位、反照率、阴影和材质,全部从分离的领域特定数据集中学习。我们设计了一种简单有效的蒸馏步骤:各任务专家通过轻量级任务投影器监督统一主干网络,无需标注重叠或伪标注。核心洞察是,预训练扩散模型的强大视觉先验足以弥合来自不同训练源的领域差异,实现对训练中未见的场景-任务组合的鲁棒泛化。UniD性能媲美专用模型与多任务基线,在分布外场景中表现优异,且具有更强的时间一致性和跨任务一致性。代码与视频结果见https://unid-video.github.io/。

原文摘要 · Abstract (English)

Scene understanding requires simultaneous prediction about geometry, appearance, and semantics. However, existing task-specific annotations are fragmented across incompatible, domain-specific datasets. Current unified systems circumvent this by restricting training to fully co-annotated data, or by incurring the large computational cost of pseudo-labeling. To mitigate this, we introduce UniD, a unified video model that jointly predicts eight dense scene properties-depth, surface normals, semantic segmentation, boundaries, human parts, albedo, shading, and materials-all learned from disjoint, domain-specific datasets. We propose a simple yet effective distillation step in which per-task experts supervise a unified backbone through lightweight task projectors, eliminating the need for annotation overlap or pseudo-labeling. Our key insight is that the strong visual priors of a pretrained diffusion model are sufficient to bridge the domain gaps introduced by disjoint training sources, enabling robust generalization to scene-task combinations never seen during training. UniD achieves competitive performance against per-task specialists and multi-task baselines, with strong generalization to out-of-distribution scenarios and enhanced temporal and cross-task consistency. Code and video results are available at https://unid-video.github.io/.

视频理解多任务学习统一模型扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。