用概率图模型自动学习任务混合比例，提升大模型微调效率
Learning Task Mixtures from Task Affinities: A Probabilistic Graphical Model for Supervised Fine-Tuning
- 构建任务间的马尔可夫随机场，通过行为差异衡量任务关系
- 在多个模型和评测集上优于均匀或按规模分配的固定混合策略
- 能揭示任务间冗余与覆盖权衡，适合需要高效微调的研究者
大语言模型的监督微调性能高度依赖于训练预算在异构任务集上的分配方式。实践中常采用固定混合策略（如均匀或按规模比例采样），忽略任务间相互作用，导致迁移效果差且浪费资源。本文提出TaskPGM框架，基于能量模型学习连续的任务混合比例。任务作为马尔可夫随机场的节点：一元势能捕捉单个任务效用，二元势能通过单任务微调模型预测分布的差异度量（如Jensen-Shannon散度、点互信息）编码任务间关系。优化该目标可获得兼顾覆盖与去重的混合方案。我们证明在预算约束下，所得集合函数为弱子模，支持离散选择的近似保证。在多个模型（LLaMA-7B, Qwen2-7B）和评测集（BIG-Bench Hard）上，TaskPGM均优于标准混合策略，并提供可解释的任务交互结构。
原文摘要 · Abstract (English)
Supervised fine-tuning performance for large language models depends strongly on how training budget is distributed across a heterogeneous set of tasks. In practice, mixtures are often fixed using simple heuristics (e.g., uniform or size-proportional sampling) that ignore task interactions, which can hurt transfer and waste budget on redundant sources. We introduce TaskPGM, a framework for learning continuous task mixtures via an energy-based model over tasks. Tasks form the nodes of a Markov random field: unary potentials capture per-task utility, and pairwise potentials encode inter-task relationships using behavioral divergences computed from predictive distributions of single-task fine-tuned models (e.g., Jensen--Shannon divergence and pointwise mutual information). Optimizing this objective yields mixtures that balance coverage against redundancy. We show that the resulting set function is weakly submodular under budget constraints, enabling approximation guarantees for discrete selection variants. Across multiple model families (LLaMA-7B, Qwen2-7B) and evaluation suites (BIG-Bench Hard), TaskPGM improves over standard mixing strategies and provides interpretable structure over task interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。