用梯度提升树做CCA编码器,兼具非线性建模与可解释性。
TreeCCA: Canonical Correlation Analysis via Gradient-Boosted Trees

- 用Eckart-Young损失实现树模型端到端训练,兼容现有工具库
- 在合成数据上性能超深网络CCA,稀疏数据下精度达1.00
- 树结构天然可解释,能验证物理假设,适合需要透明性的场景
梯度提升树在表格数据建模中占主导地位,但典型相关分析(CCA)长期依赖线性或神经网络编码器。本文提出首个将梯度提升树集成作为CCA编码器的端到端方法——TreeCCA,继承其开箱即用的可靠性:无需架构设计、使用熟悉超参数、默认配置即可取得优异性能。核心技术是Eckart-Young(EY)损失,提供闭式逐样本梯度,可直接嵌入任一标准GBT库(如XGBoost、LightGBM)作为自定义目标函数。TreeCCA首次实现了非线性准确性与原生可解释性的结合:每个树分裂选择一个特征,因此特征重要性可直接揭示驱动跨视图相关性的输入,无需额外开销。在合成基准测试中,TreeCCA在符号幂(Signed Power)指标上达到2.61,优于Deep CCA的2.43;在赫米特(Hermite)指标上为2.93,优于2.89。在零线性协方差的稀疏基准上,当特征数p=50时,TreeCCA的Precision@S达1.00,而PMD未检测到任何信号。在UCI HAR传感器融合数据集上,TreeCCA以五分之一成本达到与Deep CCA相当的准确率,且XGBoost特征重要性直接验证了基于物理的假设——这是神经编码器难以提供的可解释性。在五个主流表格多视图数据集上,TreeMCCA始终在非线性相关提取和下游分类准确率上匹配或超越线性CCA。
原文摘要 · Abstract (English)
Gradient-boosted trees dominate tabular machine learning, yet canonical correlation analysis has always relied on linear or neural encoders. We propose \textbf{TreeCCA}, the first method to train gradient-boosted tree ensembles end-to-end as CCA encoders, inheriting their plug-and-play reliability: no architecture design, familiar hyperparameters, and strong performance with defaults. The technical enabler is the Eckart-Young (EY) loss, which supplies closed-form per-sample gradients that slot directly into any standard GBT library (XGBoost, LightGBM) as a custom objective. TreeCCA is the first CCA method to combine nonlinear accuracy with native interpretability: every tree split selects one feature, so gain importances reveal which inputs drive cross-view correlation at no extra cost. We demonstrate these properties on synthetic benchmarks, where TreeCCA matches or exceeds Deep CCA (2.61 vs.\ 2.43 on Signed Power; 2.93 vs.\ 2.89 on Hermite), and on a sparse benchmark with zero linear cross-view covariance, where TreeCCA recovers the true support with $\text{Precision@}S = 1.00$ at $p=50$ while PMD finds no signal. On the UCI HAR sensor-fusion benchmark, TreeCCA achieves comparable accuracy to Deep CCA at $5\times$ lower cost, while XGBoost gain importances directly validate a physics-motivated hypothesis about the data --- an interpretation not readily available with neural encoders. Across five popular tabular multi-view datasets, TreeMCCA consistently matches or exceeds linear CCA in both nonlinear correlation extraction and downstream classification accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。