arXiv:2512.18059math.NAcs.LG2025-12被引 1

用张量列车组合函数,实现高效高维逼近。

Approximation and learning with compositional tensor trains

  • 通过低秩函数组合构建张量列车结构,可编码多项式、固定宽度神经网络等。
  • 相比传统神经网络,支持逐层可控压缩,优化时利用张量代数加速。
  • 适合需要高维建模与计算效率的机器学习任务,如回归与函数逼近。

我们提出组合张量列车(CTTs)用于多变量函数的逼近,该模型通过张量列车格式中的低秩函数组合而成。此格式能编码标准近似工具,如(稀疏)多项式、固定宽度深度神经网络(DNNs)、输入任意置换的张量网络,或更一般的仿射坐标变换,且复杂度相当。该格式可视为宽度随输入维度指数增长的神经网络,但具有结构化权重矩阵。相比传统DNN,该格式可通过高效的张量代数实现层级别的受控压缩。在优化方面,我们推导出一种受自然梯度下降启发的分层算法,利用高效的低秩张量代数。该方法依赖于格拉姆矩阵的低秩估计与张量结构随机抽样。将该格式视为离散动力系统后,还衍生出受最优控制数值方法启发的优化算法。回归任务的数值实验验证了新格式的表达能力及所提优化算法的有效性。总体而言,CTTs结合了组合模型的表达力与张量代数的算法效率,为标准深度神经网络提供了一种可扩展的替代方案。

原文摘要 · Abstract (English)

We introduce compositional tensor trains (CTTs) for the approximation of multivariate functions, a class of models obtained by composing low-rank functions in the tensor-train format. This format can encode standard approximation tools, such as (sparse) polynomials, deep neural networks (DNNs) with fixed width, or tensor networks with arbitrary permutation of the inputs, or more general affine coordinate transformations, with similar complexities. This format can be viewed as a DNN with width exponential in the input dimension and structured weights matrices. Compared to DNNs, this format enables controlled compression at the layer level using efficient tensor algebra. On the optimization side, we derive a layerwise algorithm inspired by natural gradient descent, allowing to exploit efficient low-rank tensor algebra. This relies on low-rank estimations of Gram matrices, and tensor structured random sketching. Viewing the format as a discrete dynamical system, we also derive an optimization algorithm inspired by numerical methods in optimal control. Numerical experiments on regression tasks demonstrate the expressivity of the new format and the relevance of the proposed optimization algorithms. Overall, CTTs combine the expressivity of compositional models with the algorithmic efficiency of tensor algebra, offering a scalable alternative to standard deep neural networks.

张量网络函数逼近神经网络优化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。