用强化学习自动生成辅助任务，提升主任务性能。
Generating Auxiliary Tasks with Reinforcement Learning
- 通过强化学习动态为每个样本分配辅助标签，优化主任务表现。
- 在CIFAR-100上超越人工设计的辅助任务，媲美顶级双层优化方法。
- 无需梯度回传，计算开销低，适合资源受限场景。
辅助学习（AL）是一种多任务学习形式，通过在辅助任务上训练模型以提升主任务性能。尽管已在导航、图像分类和自然语言处理等领域提升泛化能力，但传统AL依赖人力标注的辅助任务，设计成本高且需领域知识。元学习方法虽可自动生成辅助任务，但通常采用基于梯度的双层优化，带来显著计算与实现开销。本文提出RL-AUX框架，利用强化学习动态为每个训练样本分配辅助标签，当代理选择能提升主任务表现时给予奖励，并探索对每个样本的辅助损失加权。在将CIFAR-100按20个超类分组的任务上，该方法性能优于人工设计的辅助任务，且达到主流双层优化基线水平。其他分类数据集也取得类似优异结果。结果表明，强化学习是生成有效辅助任务的可行路径。
原文摘要 · Abstract (English)
Auxiliary Learning (AL) is a form of multi-task learning in which a model trains on auxiliary tasks to boost performance on a primary objective. While AL has improved generalization across domains such as navigation, image classification, and NLP, it often depends on human-labeled auxiliary tasks that are costly to design and require domain expertise. Meta-learning approaches mitigate this by learning to generate auxiliary tasks, but typically rely on gradient based bi-level optimization, adding substantial computational and implementation overhead. We propose RL-AUX, a reinforcement-learning (RL) framework that dynamically creates auxiliary tasks by assigning auxiliary labels to each training example, rewarding the agent whenever its selections improve the performance on the primary task. We also explore learning per-example weights for the auxiliary loss. On CIFAR-100 grouped into 20 superclasses, our RL method outperforms human-labeled auxiliary tasks and matches the performance of a prominent bi-level optimization baseline. We present similarly strong results on other classification datasets. These results suggest RL is a viable path to generating effective auxiliary tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。