用扩散模型训练多任务模型，仅需部分标注数据即可实现零样本学习。
StableMTL: Repurposing Latent Diffusion Models for Multi-Task Learning from Partially Annotated Synthetic Datasets
- 将扩散模型转为潜在空间回归，用统一损失替代多任务损失
- 在8个基准上7项任务表现超越基线，支持无缝扩展新任务
- 通过任务注意力机制促进跨任务共享，适合少样本多任务场景
密集预测的多任务学习受限于每项任务都需要大量标注，尽管已有研究探索使用部分标签训练。我们利用扩散模型的强大泛化能力，将部分标签学习设定拓展至零样本设置：在多个合成数据集上训练多任务模型，每个数据集仅标注了部分任务。提出的方法StableMTL将图像生成器重用于潜在空间回归，采用带有任务编码、任务条件和定制训练策略的去噪框架。不依赖需精细平衡的逐任务损失，而是采用统一的潜在空间损失，实现任务数量的无缝扩展。为促进任务间协同，引入多流模型与任务注意力机制，将N对N的任务交互转化为高效的1对N注意力，增强跨任务信息共享。StableMTL在8个基准上的7项任务中优于基线。
原文摘要 · Abstract (English)
Multi-task learning for dense prediction is limited by the need for extensive annotation for every task, though recent works have explored training with partial task labels. Leveraging the generalization power of diffusion models, we extend the partial learning setup to a zero-shot setting, training a multi-task model on multiple synthetic datasets, each labeled for only a subset of tasks. Our method, StableMTL, repurposes image generators for latent regression. Adapting a denoising framework with task encoding, per-task conditioning and a tailored training scheme. Instead of per-task losses requiring careful balancing, a unified latent loss is adopted, enabling seamless scaling to more tasks. To encourage inter-task synergy, we introduce a multi-stream model with a task-attention mechanism that converts N-to-N task interactions into efficient 1-to-N attention, promoting effective cross-task sharing. StableMTL outperforms baselines on 7 tasks across 8 benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。