解决传感误差下的频谱感知难题,提出可计算的索引策略
Restless bandits with imperfect binary feedback: PCL-indexability analysis and computation
- 基于部分保全律框架,构建信念状态下的索引可判定方法
- 在多类阈值区间内获得可解析验证的奖励与资源度量表达式
- 数值算法支持复杂场景计算,性能显著优于传统基准
我们研究具有二元隐状态和不完美二元反馈的非静止老虎机问题,动机源于存在感知误差的机会频谱接入。针对对应的信念状态模型,本文建立了一个基于部分保全律(PCL)的分析与计算框架,用于验证索引性并评估Whittle索引,该框架基于对实状态折扣非静止老虎机的验证定理。通过关联的确定性骨架、更新分解及词组合分析随机动态。在若干阈值区域内,该框架导出了折扣收益和资源度量的可处理表达式,实现对PCL索引性条件的完整验证。对于未完成解析验证的剩余区域,本文推导出高效的数值算法,用于计算相关边际度量与边际生产率(MP)索引,当条件满足时,其等于Whittle索引。大量计算实验表明,在广泛参数范围内,这些条件仍成立,且无需前人研究中的严格参数限制。实验进一步显示,MP索引策略通常显著优于标准基准策略,提升幅度可观。
原文摘要 · Abstract (English)
We study restless bandits with binary latent states and imperfect binary feedback, motivated by opportunistic spectrum access with sensing errors. For the associated belief-state model, we develop a partial conservation laws (PCL)-based analytical and computational framework for establishing indexability and evaluating the Whittle index, building on a verification theorem for real-state discounted restless bandits. The framework analyzes the stochastic dynamics via an associated deterministic skeleton, renewal decompositions, and combinatorics on words. It yields tractable expressions for discounted reward and resource metrics in several threshold regimes, enabling full verification of the PCL-indexability conditions there. For the remaining regime, where a complete analytic verification is not achieved in this paper, we derive efficient numerical schemes for computing the relevant marginal metrics and the marginal productivity (MP) index, which equals the Whittle index when those conditions hold. Extensive computational experiments provide strong evidence that these conditions also hold in that regime across broad parameter ranges and without the stringent parameter restrictions imposed in prior work. The experiments further show that theMP index policy typically outperforms standard benchmark policies, often by a substantial margin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。