用多任务优化指标自动选择最佳权重，让简单加权法效果媲美复杂方法。
AutoScale: Linear Scalarization Guided by Multi-Task Optimization Metrics
- 基于多任务优化指标，发现高效权重有特定趋势
- 在多个数据集上性能超越传统搜索方法，效率更高
- 适合追求高效多任务训练的开发者和研究者
近期多任务学习研究发现，使用合理固定任务权重的线性加权法，可达到甚至优于复杂多任务优化（MTO）方法的性能。但为何某些权重表现最优，以及如何避免耗时的超参数搜索仍不明确。本文建立线性加权与MTO方法之间的直接联系，通过大量实验揭示：表现优异的加权方案在关键MTO指标（如梯度幅值相似性）上具有特定趋势。基于此，提出AutoScale——一种无需昂贵权重搜索的两阶段框架，利用MTO指标指导权重选择，在包括新构建的大规模基准在内的多种数据集上均实现高效且优越的性能。
原文摘要 · Abstract (English)
Recent multi-task learning studies suggest that linear scalarization, when using well-chosen fixed task weights, can achieve comparable to or even better performance than complex multi-task optimization (MTO) methods. It remains unclear why certain weights yield optimal performance and how to determine these weights without relying on exhaustive hyperparameter search. This paper establishes a direct connection between linear scalarization and MTO methods, revealing through extensive experiments that well-performing scalarization weights exhibit specific trends in key MTO metrics, such as high gradient magnitude similarity. Building on this insight, we introduce AutoScale, a simple yet effective two-phase framework that uses these MTO metrics to guide weight selection for linear scalarization, without expensive weight search. AutoScale consistently shows superior performance with high efficiency across diverse datasets including a new large-scale benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。