提出新熵度量,揭示生成与检索的能耗平衡点。
Information Physics of Intelligence: Unifying Logical Depth and Entropy under Thermodynamic Constraints
- 用推导熵衡量从逻辑深度生成信息的能耗
- 发现存储与计算间的临界相变点,决定最优策略
- 为高效AI架构设计提供物理规律指导
人工智能模型的快速扩展暴露了模型容量(存储)与推理效率(计算)之间的根本矛盾。经典信息论关注传输与存储极限,但缺乏统一的物理框架来量化从压缩规律生成信息与从记忆中检索信息的热力学成本。本文提出一种理论框架,将信息处理视为从本体状态到载体状态的映射。引入新度量‘推导熵’,用于衡量从给定逻辑深度计算目标状态所需的有效功。通过分析香农熵(存储)与计算复杂度(时间/能量)的相互作用,我们证明存在一个临界相变点:低于该点时,记忆检索在热力学上更优;高于该点时,生成式计算成为最优策略。这一‘能量-时间-空间’守恒律为生成模型的效率提供了物理解释,并为下一代节能型AI架构设计提供了严格的数学边界。研究结果表明,推导熵最小化是生物与人工智能演化的主导原则。
原文摘要 · Abstract (English)
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits, it lacks a unified physical framework to quantify the thermodynamic costs of generating information from compressed laws versus retrieving it from memory. In this paper, we propose a theoretical framework that treats information processing as an enabling mapping from ontological states to carrier states. We introduce a novel metric, Derivation Entropy, which quantifies the effective work required to compute a target state from a given logical depth. By analyzing the interplay between Shannon entropy (storage) and computational complexity (time/energy), we demonstrate the existence of a critical phase transition point. Below this threshold, memory retrieval is thermodynamically favorable; above it, generative computation becomes the optimal strategy. This "Energy-Time-Space" conservation law provides a physical explanation for the efficiency of generative models and offers a rigorous mathematical bound for designing next-generation, energy-efficient AI architectures. Our findings suggest that the minimization of Derivation Entropy is a governing principle for the evolution of both biological and artificial intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。