针对多变量时间序列预测中梯度冲突问题,提出按变量分治的优化方法。
Not All Variables Agree: Reliability-Aware Variable-Wise Gradient Surgery for Multivariate Time-Series Forecasting

- 基于输出信号构建变量级梯度代理,识别并处理变量间冲突。
- 在五个模型、七数据集上平均降低3.61% MSE和2.93% MAE。
- 适合需要提升多变量预测一致性的研究者与工业应用。
在数据驱动训练中,多变量时间序列预测通常使用样本、变量和预测步长平均后的标量损失进行优化。这种平均虽简便,但优化器仅能感知聚合梯度,无法判断各变量贡献是否一致。我们直接测量变量级梯度,发现七大数据集中其成对余弦相似度平均有30.6%为负值。然而,冲突不等于损害:共享训练下64个变量中有35个表现劣于全输入单目标基线,且受损程度无法通过梯度冲突频率可靠预测。为此提出每变量手术(PV-Surgery),一种适用于缓存兼容层的优化器侧策略。一次反向传播即可从输出端信号构建变量级梯度代理,并保留逐点预测损失。可靠性感知选择机制筛选出代理和与共享梯度切片高度吻合的层。条件池化在不丢弃变量的前提下形成锚点与冲突池。共方向手术将变量或池化梯度与归一化均值对齐,并恢复输入范数以避免重新加权。在五种骨干网络、七数据集及四个预测步长上,PV-Surgery平均降低MSE 3.61%、MAE 2.93%。这表明,被均值损失掩盖的变量级结构是可利用的优化信号。
原文摘要 · Abstract (English)
In data-driven training, multivariate time-series forecasting is usually optimized with a scalar loss averaged over samples, variables, and horizons. This averaging is convenient, but the optimizer sees only the aggregated gradient, which does not reveal whether the variable-wise contributions align or oppose one another. To quantify how often this disagreement arises, we measure the variable-wise gradients directly and find that 30.6% of their pairwise cosine similarities are negative on average across seven datasets. However, conflict and harm are not the same thing. Under shared training 35 of the 64 variables do worse than a full-input single-target oracle, and the harmed fraction is not reliably predicted by how often gradients conflict. We propose Per-Variable Surgery (PV-Surgery), an optimizer-side training strategy for backbones with cache-compatible layers. One backward pass builds variable-wise gradient proxies from output-side signals and keeps the pointwise forecasting loss. Reliability-aware selection targets layers whose proxy sums closely approximate their shared-gradient slices. Conditional pooling forms anchor and conflict pools without dropping variables. Common-direction surgery aligns variable or pooled gradients with their normalized mean and restores input norms to avoid reweighting. In experiments across five backbones, seven datasets, and four horizons, PV-Surgery lowers MSE by 3.61% and MAE by 2.93% on average. For multivariate forecasting, this indicates that the variable-wise structure hidden by mean-loss training is a usable optimization signal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。