arXiv:2608.31120cs.LGcs.CC2026-09

研究条件分布兼容性难题,发现用电路编码时问题变得极难求解。

On the Complexity of the Compatibility Problem for Succinctly Encoded Conditional Distributions

  • 用算术电路压缩条件分布,研究其兼容性
  • 非零概率下为共NP完全,允许零概率时达PSPACE完全
  • 揭示了模型复杂度与可解释性的根本矛盾,适合理论研究者

本文研究机器学习中概率模型隐含的权衡问题。当使用条件概率进行预测时,一对条件分布 p(x|y) 与 p(y|x) 可能不兼容任何联合分布 p(x,y)。判断这种兼容性的问题称为兼容性问题。对于离散变量且以概率表形式编码的情况,该问题已有已知解法,计算上是可行的。本文形式化并研究了条件分布的简洁编码版本,采用算术电路表示,适用于高维场景如神经网络模型。我们证明:当条件分布以算术电路简洁表示时,兼容性问题变为不可行。在所有概率非零的情况下,问题为 co-NP 完全;在允许概率为零的情况下,多个兼容性概念可区分,且多个版本为 PSPACE 完全。此外,假设多项式层次不坍缩,则存在可兼容的简洁条件分布,其联合分布无法简洁表达。这些结果对概率建模和机器学习有深远影响。

原文摘要 · Abstract (English)

The motivation for this paper is the investigation of the trade-offs implicit in probabilistic models used in machine learning. Models are often used to make predictions in the form of conditional probabilities. However, a pair of conditional distributions p(x|y) and p(y|x) may not be compatible with any joint distribution p(x,y). Given two such conditionals, determining if there exists a compatible joint is known as the compatibility problem. For discrete random variables, when the conditionals are encoded as probability tables, the compatibility problem has a known solution, which is computationally tractable. In this paper, we formalise and study a succinct version of the problem, encoding conditional distributions as arithmetic circuits. This is applicable to practical applications of probabilistic modelling in high-dimensional settings, including neural network models. We show that, for succinct circuit representations of conditionals, the compatibility problem is intractable. In the case that all probabilities are non-zero, the problem is co-NP-complete. In the case that probabilities can be zero, we give examples to demonstrate that several notions of compatibility can be distinguished, and we prove that multiple versions of the problem are PSPACE-complete. Furthermore, we show that, assuming the polynomial hierarchy does not collapse, there exist compatible succinct conditionals whose joint cannot be expressed succinctly. Implications of these results for probabilistic modelling and machine learning are discussed.

概率建模复杂度分析算术电路

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。