arXiv:2507.03884stat.MLcs.LG2025-07被引 3

重审布雷曼的数据模型与算法模型之争,揭示现代机器学习如何突破简单性与准确性的矛盾。

Leo Breiman, the Rashomon Effect, and the Occam Dilemma

  • 对比数据模型与算法模型,指出布雷曼提出的'奥卡姆困境'在现代算力下不再成立
  • 证实'拉什蒙德效应'存在,即多个复杂模型可同样优秀,打破单一最优模型幻想
  • 主张因果推断无需依赖简单模型,解释性可独立于模型复杂度实现

在著名的《两种文化》论文中,利奥·布雷曼提出了数据模型(考虑数据生成过程)与算法模型(纯机器学习模型)的分野。本文提供现代视角:布雷曼反对数据模型的核心论点是‘拉什蒙德效应’——存在多个表现相当但结构各异的模型,导致无法确定哪个真实生成了数据;其支持数据模型的理由是‘奥卡姆困境’,即准确性与简单性不可兼得。然而,25年来的算力飞跃表明,算法模型无需复杂即可高精度,故该困境不普遍成立。值得注意的是,布雷曼所指的‘简单’可能仅限线性模型或未优化决策树。事实上,拉什蒙德效应正是证明奥卡姆困境失效的关键工具。尽管如此,布雷曼对可解释性的追求与因果关系相关——他认为简单模型能揭示变量与结果间的因果联系。但本文认为,因果分析无需依赖单一简单模型,解释性本身即具价值。技术上,本文立场既非布雷曼两文化之一,却融合了二者目标:因果、简洁、准确,并展示这些目标可通过新路径达成,摆脱原初限制。

原文摘要 · Abstract (English)

In the famous Two Cultures paper, Leo Breiman provided a visionary perspective on the cultures of ''data models'' (modeling with consideration of data generation) versus ''algorithmic models'' (vanilla machine learning models). I provide a modern perspective on these approaches. One of Breiman's key arguments against data models is the ''Rashomon Effect,'' which is the existence of many different-but-equally-good models. The Rashomon Effect implies that data modelers would not be able to determine which model generated the data. Conversely, one of his core advantages in favor of data models is simplicity, as he claimed there exists an ''Occam Dilemma,'' i.e., an accuracy-simplicity tradeoff. After 25 years of powerful computers, it has become clear that this claim is not generally true, in that algorithmic models do not need to be complex to be accurate; however, there are nuances that help explain Breiman's logic, specifically, that by ''simple,'' he appears to consider only linear models or unoptimized decision trees. Interestingly, the Rashomon Effect is a key tool in proving the nullification of the Occam Dilemma. To his credit though, Breiman did not have the benefit of modern computers, with which my observations are much easier to make. Breiman's goal for interpretability was somewhat intertwined with causality: simpler models can help reveal which variables have a causal relationship with the outcome. However, I argue that causality can be investigated without the use of single models, whether or not they are simple. Interpretability is useful in its own right, and I think Breiman knew that too. Technically, my modern perspective does not belong to either of Breiman's Two Cultures, but shares the goals of both of them - causality, simplicity, accuracy - and shows that these goals can be accomplished in other ways, without the limitations Breiman was concerned about.

模型解释因果推断机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。