提出新方法同时找关键特征与子空间,提升非平稳非线性回归的预测精度。
On a joint simultaneous learning of relevant feature subsets and subspaces in regression-like problems

- 联合优化特征子集与特征子空间,用熵最优思想提升建模效率。
- 在洛伦兹-96和哈塞加瓦-若田模型上,误差比现有方法低数个数量级。
- 适合高复杂度、小样本的物理系统建模,如等离子体与混沌系统。
我们将最近提出的熵最优流形聚类(EOMC)扩展为熵最优流形回归(EOMR),以实现对非平稳、非线性回归问题中相关特征子集与子空间的联合同时识别。所提方法具有线性增长的迭代与内存复杂度,计算高效。在极具挑战性的混沌与流体动力学问题上进行评估:(i)在强混沌与极强混沌状态下的洛伦兹-96系统($F=8$ 和 $F=12$)动态预测;(ii)托卡马克边缘等离子体的哈塞加瓦-若田模型数据。结果表明,当前最先进的机器学习与人工智能工具(包括通用梯度提升随机森林、深度神经网络及基于Transformer的TabPFN v.03)在这些任务上均表现显著逊色——预测均方根误差高出数个数量级,模型复杂度也大幅增加。对于哈塞加瓦-若田案例,EOMR成功提炼出仅含8个参数的线性、因果、弱平稳自回归过程,精确描述了主导主成分(EOF)的动力学特征。
原文摘要 · Abstract (English)
We extend a recently introduced Entropy-Optimal Manifold Clustering (EOMC) to allow for a joint simultaneous identification of subsets and subspaces of relevant features in nonstationary and nonlinear regression problems. It is shown that the proposed extension - that we coin as Entropy-Optimal Manifold Regression (EOMR) - allows a robust learning with linearly-scaling iteration and memory complexities. EOMR is compared to the most complete set of state-of-the-art tools from the Artificial Intelligence (AI) and Machine Learning (ML) that is available to the author, on the very challenging problems from chaotic and fluid dynamics: (i) on predicting the Lorenz-96 systems dynamics in strongly- and very-strongly chaotic regimes (with forcing parameter being $F=8$ and $F=12$, respectively); and, (ii) on a data from the Hasegawa-Wakatani model on the edge of the tokamak plasma. It is demonstrated that the proposed benchmarks (i) and (ii), indeed, are the very challenging problems for the state of the art ML and AI tools - since both the general-purpose gradient boosted random forests and deep neuronal networks, as well as transformer-based AI tools like TabPFN v.03 (more spezialised for large-dimensional small data learning problems) - result in orders of magnitude inferior root mean squared prediction errors, and orders of magnitude larger model complexities, when compared to the EOMR. For a Hasegawa-Wakatani example, EOMR distills a very simple entropy-optimal and skilful description of the leading Essential Orthogonal Function (EOF) dynamics, given by linear, causal and weakly-stationary autoregressive process described by just 8 parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。