梯度分析任务相似性需共享样本，否则结果无效。
Information-Theoretic Requirements for Gradient-Based Task Affinity Estimation in Multi-Task Learning
- 任务分析必须在相同输入上进行，否则梯度信号会混淆分布差异
- 样本重叠率低于30%时梯度相关性无意义，高于40%才能准确反映真实结构
- 解释了多年多任务学习结果不一致的原因，适用于生物医学数据
多任务学习表现极不稳定——有时联合训练显著提升性能，有时反而损害效果——但目前缺乏可预测结果的理论框架。我们发现梯度驱动的任务分析存在一个未明言的基本假设：任务必须共享训练样本，才能让梯度冲突揭示真实关系。当任务基于相同输入计算时，梯度对齐反映共同机制结构；若基于互不重叠的输入，则任何表面信号会混杂任务间关系与分布偏移。我们发现这一样本重叠要求存在明显相变：重叠率低于30%时，梯度-任务相关性统计上无法区分于噪声；超过40%时，能可靠恢复已知生物学结构。在多个数据集上的全面验证显示强相关性，并重构出生物通路组织。标准基准如MoleculeNet（<5%重叠）、TDC（8–14%重叠）均严重违反此条件，远低于梯度分析有效的阈值。这为过去七年多任务学习结果的不一致性提供了首个原理性解释。
原文摘要 · Abstract (English)
Multi-task learning shows strikingly inconsistent results -- sometimes joint training helps substantially, sometimes it actively harms performance -- yet the field lacks a principled framework for predicting these outcomes. We identify a fundamental but unstated assumption underlying gradient-based task analysis: tasks must share training instances for gradient conflicts to reveal genuine relationships. When tasks are measured on the same inputs, gradient alignment reflects shared mechanistic structure; when measured on disjoint inputs, any apparent signal conflates task relationships with distributional shift. We discover this sample overlap requirement exhibits a sharp phase transition: below 30% overlap, gradient-task correlations are statistically indistinguishable from noise; above 40%, they reliably recover known biological structure. Comprehensive validation across multiple datasets achieves strong correlations and recovers biological pathway organization. Standard benchmarks systematically violate this requirement -- MoleculeNet operates at <5% overlap, TDC at 8-14% -- far below the threshold where gradient analysis becomes meaningful. This provides the first principled explanation for seven years of inconsistent MTL results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。