arXiv:2606.16013cond-mat.dis-nncs.LG2026-06

线性回归在特征相关时权重不稳,导致物理意义难以解释。

The limits of interpretability in multiple linear regression

  • 通过分析特征相关矩阵的特征模态,揭示权重震荡的数学机制。
  • 小特征值模式放大权重波动,产生无实际意义的振荡模式。
  • 正则化可抑制不稳,但解读仍需谨慎,适用于物理与多领域数据。

机器学习模型的可解释性在物理科学中备受关注,人们常希望理解内在机制而非仅做预测。多重线性回归常被视为比深度神经网络更可解释的替代方案,因其预测为输入特征的加权和。然而,当特征强相关(即存在多重共线性)时,学习到的权重会表现出显著的样本间波动和跨物理相似特征的振荡行为,使解释变得困难甚至不可能。尽管统计学中已知权重在多重共线性下的不稳定性,但其对物理解释的影响,特别是与物理相似特征间的振荡权重的关系,尚未系统阐明。本文从理论上分析特征相关矩阵的特征模态,发现与多重共线性相关的低特征值模态会放大权重波动并生成非物理性的振荡模式。我们在物理数据集上进行数值验证,发现岭正则化能抑制这些不稳模态,但所得权重仍需谨慎解读。进一步在多个公开数据集上验证了结论的普适性。结果表明,即使在线性回归中,多重共线性仍可能导致物理解释难以实现。

原文摘要 · Abstract (English)

Interpreting machine-learning models has attracted increasing attention, particularly in the physical sciences, where one often seeks to understand the underlying mechanisms rather than merely make predictions. Multiple linear regression is often regarded as an interpretable alternative to more complex models, such as deep neural networks, because its predictions are expressed as explicit weighted sums of input features. However, when input features are strongly correlated, namely in the presence of multicollinearity, the learned weights can exhibit large dataset-to-dataset fluctuations and oscillatory behavior across physically similar features, making their interpretation difficult or even impossible. Although the instability of the weights under multicollinearity is well known in statistics, its consequences for physical interpretation, in particular its connection to oscillatory weights across physically similar features, have not been systematically clarified. Here, we theoretically discuss the mechanism behind this loss of interpretability by analyzing the eigenmodes of the feature correlation matrix. We show that small-eigenvalue modes associated with multicollinearity amplify fluctuations in the weights and generate oscillatory patterns that do not necessarily reflect meaningful contributions. We test this theoretical picture numerically on physics datasets and show that Ridge regularization suppresses these unstable modes, although the resulting weights must still be interpreted with caution. We further confirm the generality of our findings beyond physics by analyzing a diverse collection of publicly available datasets. Our results clarify why, in the presence of multicollinearity, physical interpretation can remain difficult even for linear regression models.

可解释性线性回归多重共线性物理建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。