用人工数据训练神经网络，让模型自动学会先验分布。
Position: The Future of Bayesian Prediction Is Prior-Fitted
- 用随机生成的数据训练网络，让模型隐含学习数据分布的先验
- 在真实数据稀缺时，能高效利用预训练算力提升小样本性能
- 适合对计算资源敏感、数据少的场景，如医疗或工业检测
在随机生成的人工数据集上训练神经网络，可使模型捕获数据生成分布所定义的先验。先验-数据拟合网络（PFNs）是一类利用这一现象的方法。在预训练算力迅速增长而真实世界数据生成近乎停滞的背景下，PFNs有望在更多领域发挥重要作用。它们能够将预训练算力高效分配到低数据场景中。最初应用于小型贝叶斯建模任务，如今已扩展至更复杂的领域和更大规模数据集。本文主张，PFNs及其他近似推理方法代表了贝叶斯推断的未来，通过近似学习应对数据稀缺问题。因此我们认为该方向极具研究价值。本文探讨其潜力及克服现有局限性的可能路径。
原文摘要 · Abstract (English)
Training neural networks on randomly generated artificial datasets yields Bayesian models that capture the prior defined by the dataset-generating distribution. Prior-data Fitted Networks (PFNs) are a class of methods designed to leverage this insight. In an era of rapidly increasing computational resources for pre-training and a near stagnation in the generation of new real-world data in many applications, PFNs are poised to play a more important role across a wide range of applications. They enable the efficient allocation of pre-training compute to low-data scenarios. Originally applied to small Bayesian modeling tasks, the field of PFNs has significantly expanded to address more complex domains and larger datasets. This position paper argues that PFNs and other amortized inference approaches represent the future of Bayesian inference, leveraging amortized learning to tackle data-scarce problems. We thus believe they are a fruitful area of research. In this position paper, we explore their potential and directions to address their current limitations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。