从生物能量原理出发,设计节能型神经网络架构。
minAction.net: Energy-First Neural Architecture Design -- From Biological Principles to Systematic Validation

- 基于能量最小化原则构建神经网络,用单一正则项控制内部激活能耗。
- 能耗降低三个数量级,准确率损失小于0.5个百分点。
- 在视觉、文本等任务中效率提升5%至33%,适合低功耗场景应用。
现代机器学习追求精度,却忽视内在计算成本,而物理与生物系统始终受能量约束。本文通过2,203组实验(每配置10个种子,因子统计分析)评估能量感知学习,覆盖视觉、文本、类脑与生理数据集。结果发现:第一,仅架构解释精度方差极小(偏eta²=0.001),但架构与数据集交互效应显著(偏eta²=0.44,p<0.001),表明最优架构依赖任务模态,否定通用最佳架构假设;第二,对lambda∈{0, 1e-5, 1e-4, 1e-3, 1e-2}的控制性扫描验证了单参数能量正则目标L = L_CE + lambda * E(theta, x):在此范围内,内部激活能耗相较无正则λ=0基线下降约三个数量级,且在MNIST与Fashion-MNIST上准确率变化低于0.5个百分点;第三,受作用量原理启发的能量优先架构,在同模态任务中训练效率较传统基线提升5%-33%。研究建立经典力学作用泛函、统计物理自由能与变分推断中KL正则目标之间的结构对应关系,将其作为设计假设而非推导结论。
原文摘要 · Abstract (English)
Modern machine learning optimizes for accuracy without explicit treatment of internal computational cost, even though physical and biological systems operate under intrinsic energy constraints. We evaluate energy-aware learning across 2,203 experiments spanning vision, text, neuromorphic, and physiological datasets with 10 seeds per configuration and factorial statistical analysis. Three findings emerge. First, architecture alone explains negligible variance in accuracy (partial eta^2 = 0.001), while the architecture x dataset interaction is large (partial eta^2 = 0.44, p < 0.001), demonstrating that optimal architecture depends critically on task modality and rejecting the assumption of a universal best architecture. Second, a controlled lambda-sweep across lambda in {0, 1e-5, 1e-4, 1e-3, 1e-2} validates a single-parameter energy-regularized objective L = L_CE + lambda * E(theta, x): across this range, internal activation energy decreases by approximately three orders of magnitude relative to the unregularized lambda=0 baseline, with negligible accuracy change (<0.5 percentage points) on both MNIST and Fashion-MNIST. Third, energy-first architectures inspired by an action-principle framework yield 5-33% within-modality training-efficiency gains over conventional baselines. These results emerge from a research program that interprets learning through a structural correspondence between the action functional in classical mechanics, free energy in statistical physics, and KL-regularized objectives in variational inference. We frame this correspondence as a design hypothesis, not a derivation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。