用强化学习优化概率电路结构，提升生成质量与效率
SymCircuit: Bayesian Structure Inference for Tractable Probabilistic Circuits via Entropy-Regularized Reinforcement Learning
- 采用熵正则强化学习替代贪心搜索，生成更优电路结构
- 在NLTCS数据集上样本效率提升10倍以上，逼近LearnSPN性能
- 支持结构、参数、叶节点的分层不确定性分析，适合可信推理场景
概率电路（PC）结构学习受限于贪心算法做出不可逆的局部最优决策。本文提出SymCircuit，将贪心搜索替换为通过熵正则强化学习训练的生成策略。在PC领域实现RL-as-inference框架，证明最优策略为温度调节的贝叶斯后验，在正则化温度与数据集大小成反比时可恢复精确后验。策略由SymFormer实现，一种带树相对自注意力的语法约束自回归Transformer，确保每一步生成均合法电路。引入选项级REINFORCE，仅对结构决策更新梯度，使信号噪声比提升，NLTCS数据集上样本效率提高超10倍。基于PC输出的多重线性多项式结构，构建三层不确定性分解（结构通过模型平均，参数通过delta方法，叶节点通过共轭狄利克雷-类别传播）。在NLTCS上，SymCircuit缩小了93%与LearnSPN的差距；在含69个变量的Plants数据集上的初步结果表明其具备可扩展性。
原文摘要 · Abstract (English)
Probabilistic circuit (PC) structure learning is hampered by greedy algorithms that make irreversible, locally optimal decisions. We propose SymCircuit, which replaces greedy search with a learned generative policy trained via entropy-regularized reinforcement learning. Instantiating the RL-as-inference framework in the PC domain, we show the optimal policy is a tempered Bayesian posterior, recovering the exact posterior when the regularization temperature is set inversely proportional to the dataset size. The policy is implemented as SymFormer, a grammar-constrained autoregressive Transformer with tree-relative self-attention that guarantees valid circuits at every generation step. We introduce option-level REINFORCE, restricting gradient updates to structural decisions rather than all tokens, yielding an SNR (signal to noise ratio) improvement and >10 times sample efficiency gain on the NLTCS dataset. A three-layer uncertainty decomposition (structural via model averaging, parametric via the delta method, leaf via conjugate Dirichlet-Categorical propagation) is grounded in the multilinear polynomial structure of PC outputs. On NLTCS, SymCircuit closes 93% of the gap to LearnSPN; preliminary results on Plants (69 variables) suggest scalability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。