用离散比特串表示实现高效概率推理，兼顾精度与可解释性。
Approximate Bayesian Inference via Bitstring Representations
- 将量化参数空间视为离散概率模型，用概率电路实现可训练推理
- 在2D分布和量化神经网络上验证，保持精度的同时提升推理效率
- 适合需要高可解释性与低资源部署的机器学习场景
机器学习领域近期致力于使用量化或低精度算术来扩展大规模模型。本文提出在量化离散参数空间中进行概率推理，有效利用离散参数学习连续分布。研究涵盖二维密度函数与量化神经网络，引入基于概率电路的可计算学习方法。该方法提供了管理复杂分布的可扩展方案，并为模型行为提供清晰洞察。通过多种模型验证，证明其在不牺牲精度的前提下显著提升推理效率。本工作通过离散近似实现概率计算，推动了可扩展且可解释的机器学习发展。
原文摘要 · Abstract (English)
The machine learning community has recently put effort into quantized or low-precision arithmetics to scale large models. This paper proposes performing probabilistic inference in the quantized, discrete parameter space created by these representations, effectively enabling us to learn a continuous distribution using discrete parameters. We consider both 2D densities and quantized neural networks, where we introduce a tractable learning approach using probabilistic circuits. This method offers a scalable solution to manage complex distributions and provides clear insights into model behavior. We validate our approach with various models, demonstrating inference efficiency without sacrificing accuracy. This work advances scalable, interpretable machine learning by utilizing discrete approximations for probabilistic computations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。