arXiv:2506.15492cs.LGstat.ML2025-06

通过潜在变量建模交互项,提升高维线性预测的准确性和可解释性。

LIT-LVM: Structured Regularization for Interaction Terms in Linear Predictors using Latent Variable Models

  • 用低维潜在向量表示特征,约束交互项系数结构。
  • 在交互项远多于样本时,预测精度优于弹性网等方法。
  • 生成可可视化特征关系的低维表示,适合高维数据分析。

统计与机器学习中许多常用预测器基于特征加权线性组合。通过引入特征两两乘积的交互项,线性预测器可建模非线性关系。本文关注交互项系数的准确估计问题,提出假设不同交互项系数具有近似低维结构,并将每个特征用低维空间中的潜在向量表示。该方法可视为一种结构化正则化,相较于Lasso、弹性网等标准正则化,在高维场景下进一步缓解过拟合。实验表明,所提方法LIT-LVM在多种模拟与真实数据上均优于弹性网、层级Lasso和因子分解机,尤其当交互项数量远超样本数时表现更优。此外,该方法还为特征生成低维潜在表示,有助于可视化与分析特征间关系。

原文摘要 · Abstract (English)

Some of the simplest, yet most frequently used predictors in statistics and machine learning use weighted linear combinations of features. Such linear predictors can model non-linear relationships between features by adding interaction terms corresponding to the products of all pairs of features. We consider the problem of accurately estimating coefficients for interaction terms in linear predictors. We hypothesize that the coefficients for different interaction terms have an approximate low-dimensional structure and represent each feature by a latent vector in a low-dimensional space. This low-dimensional representation can be viewed as a structured regularization approach that further mitigates overfitting in high-dimensional settings beyond standard regularizers such as the lasso and elastic net. We demonstrate that our approach, called LIT-LVM, achieves superior prediction accuracy compared to the elastic net, hierarchical lasso, and factorization machines on a wide variety of simulated and real data, particularly when the number of interaction terms is high compared to the number of samples. LIT-LVM also provides low-dimensional latent representations for features that are useful for visualizing and analyzing their relationships.

线性模型交互项潜在变量正则化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。