arXiv:2410.12389cs.AI2024-10NeurIPS被引 4

用张量与快速傅里叶变换加速整数概率推理,实现高效可微学习。

A Fast Convoluted Story: Scaling Probabilistic Inference for Integer Arithmetic

  • 将整数随机变量加法转化为对数域上的快速傅里叶变换
  • 推理与学习速度相比旧方法提升数个数量级
  • 适合需要高效概率推理的神经符号系统研究者

如整数线性规划的成功所示,线性整数逻辑是建模组合问题的强大工具。此外,线性规划的概率扩展已被用于神经符号人工智能中的问题建模。然而,两个关键问题阻碍了神经符号技术在真实问题中的应用:首先,概率推理本质上是#P难的;其次,整数的离散性使得有意义梯度的构建变得困难,这对学习不利。为缓解这些问题,我们提出将整数随机变量上的线性算术形式化为张量操作,可直接通过现代深度学习库实现。核心思想是:两个整数随机变量的加法可通过将快速傅里叶变换适配至对数域概率上完成。通过依赖张量运算,我们获得一个可微的数据结构,几乎免费地解锁了基于梯度的学习能力。实验验证表明,将概率线性整数算术张量化并利用快速傅里叶变换,使推理和学习时间相比现有方法提升数个数量级。

原文摘要 · Abstract (English)

As illustrated by the success of integer linear programming, linear integer arithmetic is a powerful tool for modelling combinatorial problems. Furthermore, the probabilistic extension of linear programming has been used to formulate problems in neurosymbolic AI. However, two key problems persist that prevent the adoption of neurosymbolic techniques beyond toy problems. First, probabilistic inference is inherently hard, #P-hard to be precise. Second, the discrete nature of integers renders the construction of meaningful gradients challenging, which is problematic for learning. In order to mitigate these issues, we formulate linear arithmetic over integer-valued random variables as tensor manipulations that can be implemented in a straightforward fashion using modern deep learning libraries. At the core of our formulation lies the observation that the addition of two integer-valued random variables can be performed by adapting the fast Fourier transform to probabilities in the log-domain. By relying on tensor operations we obtain a differentiable data structure, which unlocks, virtually for free, gradient-based learning. In our experimental validation we show that tensorising probabilistic linear integer arithmetic and leveraging the fast Fourier transform allows us to push the state of the art by several orders of magnitude in terms of inference and learning times.

概率推理整数算术可微学习张量计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。