用历史数据和潜变量模型,让碳封存系统自动适应异常情况。
Closed-Loop CO2 Storage Control With History-Based Reinforcement Learning and Latent Model-Based Adaptation

- 基于历史井数据训练强化学习控制器,不依赖理想观测
- 潜变量模型重调可比直接重训更高效应对泄漏等异常
- 适合需要长期自适应的地质碳封存运维场景
地质碳封存的闭环管理需在不确定性下做出决策,且依赖实际运行中可获取的观测数据。本文将二氧化碳注入与卤水产出控制建模为部分可观测的序列决策问题,采用高保真油藏模拟器训练深度强化学习控制器。通过对比特权状态、仅井口信息、历史条件、掩码课程和非对称师生等无模型策略,量化了时间井响应信息与训练期特权状态的价值。进一步评估了基于潜变量模型的适应性流程:在已知注气井故障、泄漏引起的动态变化、奖励偏移及储层连通性分区等异常情况下,复用原始潜变量动力学并微调控制器。结果表明,历史条件策略仅使用可部署的井级信息即可恢复近似特权状态性能;而潜变量模型重调在相同实机模拟预算下优于直接无模型重调。该框架为闭环碳封存控制提供了兼顾模拟预算的替代方案,避免重复在线历史匹配与重优化。
原文摘要 · Abstract (English)
Closed-loop management of geological CO2 storage requires control policies that adapt to uncertain reservoir behavior while relying on observations that are realistically available during operation. This work formulates CO2 injection and brine-production control as a partially observable sequential decision problem and studies deployable deep reinforcement-learning controllers trained with high-fidelity reservoir simulation. We first compare privileged-state, well-only, history-conditioned, masking-curriculum, and asymmetric teacher-student model-free policies in order to quantify the value of temporal well-response information and training-time privileged simulator states. We then evaluate a latent model-based adaptation pipeline that reuses nominal latent dynamics and retunes controllers under known injector failure, leakage-induced dynamics and reward shift, and compartmentalized reservoir connectivity. The results show that history-conditioned policies recover nearly all of the privileged-state performance while using only deployable well-level information, and that latent model-based retuning outperforms direct model-free retuning under the same scenario-specific real-simulator budget in the abnormal operating cases. The proposed framework therefore provides a simulator-budget-aware alternative to repeated online history matching and re-optimization for closed-loop CO2 storage control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。