用原型记忆动态适配多任务图像密集预测，提升精度与效率
Task-State Adaptation with Prototype Memory for Multi-Task Dense Prediction
- 从全局视觉上下文提取任务状态，通过可学习原型记忆优化
- 结合稀疏路由与残差路径，共享专家池实现高效多任务推理
- 适用于需灵活适配的视觉密集预测场景，如语义分割
视觉基础模型为密集预测提供了强大表征，但单一共享特征难以满足不同任务的图像依赖性适应需求。本文提出 MemMTL 框架，从全局视觉上下文估计紧凑的任务状态，并通过可学习的任务状态原型记忆进行优化。优化后的状态转化为任务条件专家得分，与词元级得分结合后，在所有任务共享的局部专家池中进行稀疏 top-$k$ 选择。一个独立的任务无关残差池提供通用适配路径，两条路径在融合到骨干特征前分别处理，最终完成任务特定预测。我们在 NYUD-v2 与 PASCAL-Context 数据集上，使用 SAM 3 和 ViT-L 骨干，设计了匹配的评估协议，以衡量预测质量、计算成本，以及任务状态调控、原型检索和稀疏路由的贡献。当前工作稿中的数值记录早于这一规范实现，须重新生成以支撑实证结论。
原文摘要 · Abstract (English)
Vision foundation backbones provide strong representations for dense prediction, yet a single shared feature still needs to support tasks with different, image-dependent adaptation requirements. We propose MemMTL, a multi-task dense prediction framework that estimates a compact task state from global visual context and refines it through a learnable task-state prototype memory. The refined state is converted into task-conditioned expert logits and combined with token-level logits before sparse top-$k$ selection over a local expert bank shared by all tasks. A separate task-agnostic residual bank provides a common adaptation path, and both paths are added once to the backbone feature before task-specific prediction. We specify a matched evaluation protocol on NYUD-v2 and PASCAL-Context with SAM 3 and ViT-L backbones to measure predictive quality, computational cost, and the contributions of task-state conditioning, prototype retrieval, and sparse routing. The numerical record in the present working draft predates this canonical implementation and must be regenerated before it can support empirical claims.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。