TADFormer通过动态适配任务上下文，实现高效多任务学习。
TADFormer : Task-Adaptive Dynamic Transformer for Efficient Multi-Task Learning
- 引入动态任务过滤器，根据输入上下文自适应调整特征
- 在PASCAL-Context上参数量减少8.4倍，精度更高
- 适合资源受限下需高精度多任务视觉模型的场景
迁移学习推动了视觉任务的显著进步。然而，随着模型规模增大，传统全量微调在多任务学习（MTL）中变得计算成本过高，训练复杂度随任务数线性增长。为此，近期研究探索了参数高效微调（PEFT）方法。但现有方法仍难以捕捉精细、任务特异的特征。本文提出任务自适应动态变压器TADFormer，一种新型PEFT框架，通过动态考虑任务特定输入上下文，实现细粒度特征适配。TADFormer采用参数高效的提示机制与动态任务滤波器（DTF），以输入上下文为条件捕获任务信息。在PASCAL-Context基准测试中，该方法在密集场景理解任务中达到更高精度，同时相比全量微调减少多达8.4倍可训练参数。TADFormer在参数效率和精度上均优于现有PEFT方法。
原文摘要 · Abstract (English)
Transfer learning paradigm has driven substantial advancements in various vision tasks. However, as state-of-the-art models continue to grow, classical full fine-tuning often becomes computationally impractical, particularly in multi-task learning (MTL) setup where training complexity increases proportional to the number of tasks. Consequently, recent studies have explored Parameter-Efficient Fine-Tuning (PEFT) for MTL architectures. Despite some progress, these approaches still exhibit limitations in capturing fine-grained, task-specific features that are crucial to MTL. In this paper, we introduce Task-Adaptive Dynamic transFormer, termed TADFormer, a novel PEFT framework that performs task-aware feature adaptation in the fine-grained manner by dynamically considering task-specific input contexts. TADFormer proposes the parameter-efficient prompting for task adaptation and the Dynamic Task Filter (DTF) to capture task information conditioned on input contexts. Experiments on the PASCAL-Context benchmark demonstrate that the proposed method achieves higher accuracy in dense scene understanding tasks, while reducing the number of trainable parameters by up to 8.4 times when compared to full fine-tuning of MTL models. TADFormer also demonstrates superior parameter efficiency and accuracy compared to recent PEFT methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。