arXiv:2603.01283cs.AIcs.LG2026-03

提出新指标衡量智能体与环境互动效率,揭示代理行为的内在信息代价。

The Informational Cost of Agency: A Bounded Measure of Interaction Efficiency for Deployed Reinforcement Learning

  • 引入双预测性指标P,量化智能体与环境互动中不确定性转化为共同可预测性的效率。
  • 实测21个连续控制智能体的P值为0.33±0.02,低于理论上限0.5,验证信息成本存在。
  • 该指标可跨模型通用,适用于语言、视觉和机械系统,适合部署系统运行时监控。

已部署的强化学习系统缺乏可靠的运行时可靠性理论。本文提出双预测性(Bipredictability, P),一个闭式的信息论度量,用于量化智能体与环境在闭环交互中将不确定性转化为共享可预测性的效率。P具有可证明的经典上界0.5,由香农熵的次可加性推导得出;必要响应的代理行为必然使P低于此上限,这一结构性预测称为“代理的信息代价”。在21个训练好的连续控制智能体中,我们实证确认该预测,测得P = 0.33 ± 0.02。相同抑制特征也出现在语言模型对话、卷积视觉系统和经典力学基线中,表明P捕捉的是代理互动的底层数学特性,而非算法特异性。信息数字孪生(IDT)是一种模型无关架构,能从外部交互流中计算P,相比基于奖励的监测,其对耦合退化的检测率达89.3%,而后者仅44.0%,且延迟降低4.4倍。P为部署自主系统提供了缺失的运行时可靠性与闭环自调节测量层。

原文摘要 · Abstract (English)

Deployed reinforcement learning systems lack a principled runtime reliability theory. We close this gap by introducing Bipredictability, P, a closed form information theoretic metric that quantifies how efficiently a closed loop interaction between agent and environment converts uncertainty into shared predictability. P admits a provable classical bound P equal, smaller than 0.5, derived from Shannon entropy subadditivity, and responsive agency necessarily suppresses P below this ceiling, a structural prediction we term the informational cost of agency. Across 21 trained continuous control agents, we confirm this prediction empirically at P = 0.33 plus minus 0.02. The same suppression signature reproduces in language model dialogue, convolutional vision systems, and classical mechanical baselines, indicating that P captures a substrate independent property of agentic interaction rather than an algorithm specific artifact. The Information Digital Twin, IDT, a model agnostic architecture that computes P from the external interaction stream, detects 89.3% of coupling degradations against 44.0% for reward based monitoring, with 4.4 times lower latency. P provides the missing measurement layer for runtime reliability and closed loop self regulation in deployed autonomous systems.

强化学习信息论系统监控智能体效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。