arXiv:2503.18958cs.AImath.PR2025-03

把深度学习中的概率分布当工程对象改造,让模型更可靠、灵活、高效。

Advancing Deep Learning through Probability Engineering: A Pragmatic Paradigm for Modern AI

  • 将已学概率分布视为可改造的工程构件,主动优化其性能。
  • 在贝叶斯深度学习、边缘AI和生成式AI中显著提升模型鲁棒性与适应性。
  • 适合追求可信、高效、可迭代AI系统的研究者与工程师。

近年来深度学习快速发展,推动我们向通用人工智能(AGI)迈进。概率建模在诸多进展中起关键作用,为捕捉数据分布提供基础框架。然而,随着AI应用规模与复杂度提升,传统概率建模面临高维参数空间、异构数据源及动态现实需求等挑战,经典方法逐渐显现出灵活性不足。本文提出一种新范式——概率工程(Probability Engineering),将深度学习中已学习的概率分布视作工程化成果,不再仅限于拟合或推断,而是主动修改与强化,以更好应对现代AI多样且动态的需求。具体而言,该范式引入新技巧与约束,优化现有概率分布的鲁棒性、效率、适应性与可信度。通过一系列应用案例,涵盖贝叶斯深度学习、边缘AI(包括联邦学习与知识蒸馏)、生成式AI(如基于扩散模型的文生图与大语言模型高质量文本生成),展示了原本被视为静态对象的概率分布如何被工程化以满足大规模、数据密集型、可信的AI系统需求。通过系统拓展概率建模的角色,概率工程为当今快速发展的AI时代提供了更鲁棒、自适应、高效、可信的深度学习解决方案。

原文摘要 · Abstract (English)

Recent years have witnessed the rapid progression of deep learning, pushing us closer to the realization of AGI (Artificial General Intelligence). Probabilistic modeling is critical to many of these advancements, which provides a foundational framework for capturing data distributions. However, as the scale and complexity of AI applications grow, traditional probabilistic modeling faces escalating challenges, such as high-dimensional parameter spaces, heterogeneous data sources, and evolving real-world requirements often render classical approaches insufficiently flexible. This paper proposes a novel concept, Probability Engineering, which treats the already-learned probability distributions within deep learning as engineering artifacts. Rather than merely fitting or inferring distributions, we actively modify and reinforce them to better address the diverse and evolving demands of modern AI. Specifically, Probability Engineering introduces novel techniques and constraints to refine existing probability distributions, improving their robustness, efficiency, adaptability, or trustworthiness. We showcase this paradigm through a series of applications spanning Bayesian deep learning, Edge AI (including federated learning and knowledge distillation), and Generative AI (such as text-to-image generation with diffusion models and high-quality text generation with large language models). These case studies demonstrate how probability distributions once treated as static objects can be engineered to meet the diverse and evolving requirements of large-scale, data-intensive, and trustworthy AI systems. By systematically expanding and strengthening the role of probabilistic modeling, Probability Engineering paves the way for more robust, adaptive, efficient, and trustworthy deep learning solutions in today's fast-growing AI era.

概率建模深度学习可信AI工程化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。