arXiv:2607.04019eess.SYcs.AI2026-07被引 1

为决策系统设计可信赖的信念表示,确保噪声下动作价值稳定。

Finite Reliability Representations: Noise-Calibrated Belief-Space Covers for Reliable Decision-Making

论文配图:Finite Reliability Representations: Noise-Calibrated Belief-Space Covers for Reliable Decision-Making
图 1 · 摘自论文原文
  • 用可靠性单元覆盖信念空间,保证动作价值变化不超过容忍度
  • 提出可靠性熵衡量信念复杂度,理论证明子优性上界为2ε/(1−γ)
  • 适用于非线性系统、部分可观测马尔可夫决策过程等场景

物理感知与执行存在噪声底限,应决定决策系统可依赖的信念分辨率。本文提出有限可靠性表示(FRR)框架,通过可靠性单元覆盖信念空间:在这些区域内,最优动作价值函数Q*(b,u)对所有动作的变化不超过容忍度ε。该框架基于信念而非状态构建,并采用覆盖而非等价类划分,因为近似动作相近性通常不具有传递性。关键技术点在于:不应将噪声贝叶斯更新视为对任意信念的全局收缩。因此,我们区分三个对象:固定观测滤波映射、预测观测分布和受控信念转移核。对于连续状态非线性系统,FRR 在信念转移核的可达集Lipschitz模条件下成立。对于有限状态POMDP,同一构造在信念单纯形上是精确的:预测为线性,贝叶斯修正为归一化的正线性映射,传感器噪声通过观测分布可区分性体现,执行不确定性通过动作执行信道体现。在相应动作价值Lipschitz条件下,一个单元恒定策略的子优性被限定在2ε/(1−γ)内。我们还引入可靠性熵,即最小可靠性单元数的对数,作为认证决策相关信念复杂度的度量。该框架区分了表示充分性与由感知、过程和执行噪声决定的根本性能下限。适用于有限POMDP、线性高斯滤波器、局部线性化非线性滤波器以及粒子滤波实现,可通过分析或经验验证可靠性单元。

原文摘要 · Abstract (English)

Physical sensing and actuation noise floors should inform how much belief resolution a decision-making system can reliably use. We introduce Finite Reliability Representations (FRR), a framework for covering belief spaces by reliability cells: regions within which the optimal action-value function Q*(b,u) varies by at most a tolerance epsilon, uniformly over actions. The framework is formulated on beliefs rather than states and uses a cover rather than an equivalence quotient, because approximate decision-closeness is not transitive in general. A central technical point is that noisy Bayesian updates should not be treated as globally contractive on arbitrary beliefs. We therefore separate three objects: the fixed-observation filter map, the predictive observation law, and the controlled belief-transition kernel. For nonlinear continuous-state systems, FRR is obtained under a reachable-set Lipschitz modulus for the belief-transition kernel. For finite-state POMDPs, the same construction becomes exact on the belief simplex: prediction is linear, Bayesian correction is a normalized positive linear map, sensor noise enters through observation-distribution distinguishability, and actuation uncertainty enters through an action-execution channel. Under the corresponding action-value Lipschitz condition, an FRR cover supports a cell-constant policy whose suboptimality is bounded by 2 epsilon/(1 - gamma). We also introduce reliability entropy, the logarithm of the minimal number of reliability cells, as a measure of certified decision-relevant belief complexity. The framework distinguishes representation sufficiency from fundamental performance floors imposed by sensing, process, and actuation noise. It applies to finite POMDPs, linear-Gaussian filters, locally linearized nonlinear filters, and particle-filter implementations through analytic or empirical certification of reliability cells.

决策可靠性信念空间强化学习不确定性建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。