用语义约束替代传统失真约束,扩展香农信息论以支持语义通信。
A Semantic Generalization of Shannon's Information Theory and Applications
- 以真值函数构建语义信道,将失真约束替换为语义约束
- 最大语义信息准则等价于最大似然与正则化最小二乘
- 适用于机器学习、投资组合、贝叶斯确认等场景,具跨领域潜力
本文主张通过广义化香农信息论(简称G理论)来支持语义通信,而非另建独立理论。核心思想是用语义约束替代传统失真约束,借助一组真值函数实现语义失真、语义信息度量和语义信息损失的表达。最大语义信息准则等价于最大似然准则,且与正则化最小二乘准则相似。该理论应用于日常与电子语义通信、机器学习(多标签学习、分类、混合模型、隐变量求解)、约束控制、贝叶斯确认、投资组合理论及信息价值分析。统计物理视角揭示:香农信息类比自由能,语义信息类比局部平衡系统的自由能,信息效率类比自由能做功的效率。论文进一步提出将弗里斯顿最小自由能原理优化为最大信息效率原理。最后对比了与其他语义信息理论的差异,并指出其在复杂数据语义表征上的局限性。
原文摘要 · Abstract (English)
Does semantic communication require a semantic information theory parallel to Shannon's information theory, or can Shannon's work be generalized for semantic communication? This paper advocates for the latter and introduces a semantic generalization of Shannon's information theory (G theory for short). The core idea is to replace the distortion constraint with the semantic constraint, achieved by utilizing a set of truth functions as a semantic channel. These truth functions enable the expressions of semantic distortion, semantic information measures, and semantic information loss. Notably, the maximum semantic information criterion is equivalent to the maximum likelihood criterion and similar to the Regularized Least Squares criterion. This paper shows G theory's applications to daily and electronic semantic communication, machine learning, constraint control, Bayesian confirmation, portfolio theory, and information value. The improvements in machine learning methods involve multilabel learning and classification, maximum mutual information classification, mixture models, and solving latent variables. Furthermore, insights from statistical physics are discussed: Shannon information is similar to free energy; semantic information to free energy in local equilibrium systems; and information efficiency to the efficiency of free energy in performing work. The paper also proposes refining Friston's minimum free energy principle into the maximum information efficiency principle. Lastly, it compares G theory with other semantic information theories and discusses its limitation in representing the semantics of complex data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。