用热力学思想提升异构数据建模,让模型更懂复杂数据间的差异。
ZENN: A Thermodynamics-Inspired Computational Framework for Heterogeneous Data-Driven Modeling
- 引入内在熵概念,让网络同时学习能量与熵,捕捉多源数据结构
- 在CIFAR-10/100等数据集上超越现有模型,尤其擅长预测高阶导数
- 可应用于材料科学,精准重建铁铂合金的能量景观
传统基于熵的方法(如分类中的交叉熵损失)长期用于刻画数据不确定性与物理无序性。然而,跨领域数据快速增长带来了异构数据融合的新挑战。为此,我们提出一种基于泽熵理论的增强型神经网络(ZENN),通过内在熵将热力学思想引入数据科学,实现对多源异构数据的有效学习。ZENN 同时学习能量与内在熵成分,捕获数据底层结构,并重构神经网络架构以反映数据内在属性与变异性。在分类任务与能量景观重建中,ZENN 展现出卓越泛化能力与鲁棒性,尤其在预测高阶导数方面表现突出。其引入可学习温度变量,有效建模隐含的多源异质性,在 CIFAR-10/100、BBCNews 与 AGNews 数据集上均超越当前最优模型。作为实际应用,我们使用 ZENN 基于密度泛函理论(DFT)生成的数据,重建了 Fe₃Pt 的亥姆霍兹能量景观,成功捕捉负热膨胀及温度-压力空间中的临界点等关键材料行为。本工作构建了一个基于泽熵的数据驱动学习框架,使 ZENN 成为处理复杂异构数据科学问题的通用且稳健方法。
原文摘要 · Abstract (English)
Traditional entropy-based methods - such as cross-entropy loss in classification problems - have long been essential tools for representing the information uncertainty and physical disorder in data and for developing artificial intelligence algorithms. However, the rapid growth of data across various domains has introduced new challenges, particularly the integration of heterogeneous datasets with intrinsic disparities. To address this, we introduce a zentropy-enhanced neural network (ZENN), extending zentropy theory into the data science domain via intrinsic entropy, enabling more effective learning from heterogeneous data sources. ZENN simultaneously learns both energy and intrinsic entropy components, capturing the underlying structure of multi-source data. To support this, we redesign the neural network architecture to better reflect the intrinsic properties and variability inherent in diverse datasets. We demonstrate the effectiveness of ZENN on classification tasks and energy landscape reconstructions, showing its superior generalization capabilities and robustness-particularly in predicting high-order derivatives. ZENN demonstrates superior generalization by introducing a learnable temperature variable that models latent multi-source heterogeneity, allowing it to surpass state-of-the-art models on CIFAR-10/100, BBCNews, and AGNews. As a practical application in materials science, we employ ZENN to reconstruct the Helmholtz energy landscape of Fe$_3$Pt using data generated from density functional theory (DFT) and capture key material behaviors, including negative thermal expansion and the critical point in the temperature-pressure space. Overall, this work presents a zentropy-grounded framework for data-driven machine learning, positioning ZENN as a versatile and robust approach for scientific problems involving complex, heterogeneous datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。