arXiv:2604.23716cs.AIcs.IT2026-04

教你在AI中正确选择信息论指标,避免误用陷阱。

Information-Theoretic Measures in AI: A Practical Decision Framework

论文配图:Information-Theoretic Measures in AI: A Practical Decision Framework
图 1 · 摘自论文原文
  • 按数据类型和维度匹配合适估计器,明确测量目标
  • 揭示各指标最危险的误用场景,防范推断错误
  • 提供可操作流程图与决策表,适合算法设计者参考

信息论(IT)度量在人工智能中广泛应用:熵驱动决策树分裂与不确定性量化,交叉熵是分类任务的默认损失函数,互信息支撑表示学习与特征选择,转移熵揭示动态系统中的有向影响。尽管应用广泛,但度量选择常脱离估计器假设、失效模式与安全推断声明。本文提出一个实用决策框架,针对四种基础度量——熵、KL散度/交叉熵、互信息、转移熵——围绕三个核心问题展开:(i) 度量回答什么问题,适用于哪些AI场景;(ii) 针对数据类型与维度应选用何种估计器;(iii) 最致命的误用是什么。框架通过两个互补工具实现:度量选择流程图与主决策表。覆盖机器学习与决策代理应用领域,每项度量附标准化桥接注释,链接至认知与神经科学概念。两个实例展示其在表示学习与时间影响分析中的应用;一项可复现的多智能体案例研究验证了转移熵替代检验作为零控制下的防护机制有效性。

原文摘要 · Abstract (English)

Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation learning and feature selection, and transfer entropy reveals directed influence in dynamical systems. Despite wide adoption, measure selection is often decoupled from estimator assumptions, failure modes, and safe inferential claims. This survey provides a practical decision framework for four foundational measures - Entropy, KL divergence/cross-entropy, Mutual Information, and Transfer Entropy - organized around three prescriptive questions for each: (i) what question does the measure answer and in which AI context; (ii) which estimator is appropriate for the data type and dimensionality; and (iii) what is the most dangerous misuse. The framework is operationalized in two complementary artifacts: a measure-selection flowchart and a master decision table. We cover both AI/ML and decision-making agent application domains per measure, with standardized Bridge notes linking IT quantities to cognitive and neuroscientific constructs. Two worked examples illustrate the framework on concrete practitioner scenarios spanning representation learning and temporal influence analysis, and a reproducible multi-agent case study across three learning architectures validates the transfer-entropy surrogate-testing guardrail against a null control.

信息论度量选择决策框架可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。