针对动态突变的连续控制问题，提出贝叶斯自适应鲁棒强化学习方法。
BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous control
- 融合贝叶斯在线变化检测与鲁棒集成强化学习，实现动态环境下的智能权衡。
- 理论证明其算子为收缩映射，切换后恢复时间受误差预算约束，且可形式化验证。
- 无需部署时模式标签，适合工业级非平稳控制场景，如机器人操作、自动驾驶。
现实中的控制系统常处于分段平稳状态，即长时间稳定后突然发生模式切换。传统鲁棒强化学习面临根本矛盾：全局保守策略在稳定期性能低下，而局部自适应策略在未检测到切换时可能灾难性失效。本文提出BAPR（贝叶斯遗忘分段鲁棒SAC），将贝叶斯在线变化检测（BOCD）与鲁棒集成强化学习结合。BAPR算子为模式条件贝尔曼算子的凸组合，权重由冻结信念分布决定，是γ-收缩算子。反例经Lean 4机器验证，表明当信念依赖Q函数时，收缩因子变为γ+λΔ（Δ为模式间奖励差距），且当γ+λΔ≥1时收缩性失效。推导出抽象算子的逐分量形式误差预算，所有组件均经机器验证，适用于抽象模式混合算子，并通过冻结参数设计传递至共享评论家算法。全部结果以无‘sorry’形式完成验证（共1,145行，3个Lean 4文件，22个定理）。BOCD驱动自适应保守机制：检测到变化点后政策最大化保守，随置信度上升平滑放松，检测延迟为O(log(1/δ))。训练时通过RMDM损失学习上下文编码模块，利用模拟器提供的模式ID生成模式感知表示，部署时不需模式标签。
原文摘要 · Abstract (English)
Real-world control systems frequently operate under \emph{piecewise stationary} conditions, where dynamics remain stable for extended periods before undergoing abrupt regime changes. Standard robust RL methods face a fundamental dilemma: a globally conservative policy wastes performance during stable periods, while a locally adaptive policy risks catastrophic failure when the regime changes undetected. We propose \textbf{BAPR} (Bayesian Amnesic Piecewise-Robust SAC), which unifies Bayesian Online Change Detection (BOCD) with robust ensemble RL. The BAPR operator -- a convex combination of mode-conditional Bellman operators weighted by a frozen belief distribution -- is a $γ$-contraction. A complementary counterexample, machine-verified in Lean~4, establishes a \emph{sharp boundary}: when beliefs depend on the Q-function, the contraction factor becomes $γ+ λΔ$ (where $Δ$ is the mode reward gap), and contraction fails exactly when $γ+ λΔ\geq 1$. We derive a \emph{component-wise} formal error budget for the abstract operator -- every component machine-verified -- bounding post-switch recovery; the budget applies to the abstract mode-mixture operator and inherits to the implemented shared-critic algorithm only through the frozen-parameter design intuition. All results are formally verified with no \texttt{sorry} (1,145 lines across 3 Lean~4 files, 22 machine-verified theorems). BOCD drives an adaptive conservatism mechanism: the policy becomes maximally conservative after detected change-points and smoothly relaxes as confidence grows, with detection delay $O(\log(1/δ))$. A context-conditioning module trained via RMDM loss provides mode-aware representations from simulator-provided mode IDs at training time and requires no mode labels at deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。