用强化学习动态调整水印强度,防篡改又省能耗。
DynaMark: A Reinforcement Learning Framework for Dynamic Watermarking in Industrial Machine Tool Controllers
- 将水印设计建模为马尔可夫决策过程,在线自适应调整水印方差。
- 水印能耗降低70%,检测延迟仅一个采样周期。
- 无需系统模型知识,适合工业控制器等复杂场景。
工业4.0中高度网络化的机床控制器(MTCs)易受重放攻击,即利用过时传感器数据操控执行器。动态水印可揭示此类篡改,但现有方案假设线性高斯动态且使用恒定水印统计,难以应对MTCs时变、部分专有的行为特征。本文提出DynaMark,一种基于强化学习的动态水印框架,将动态水印建模为马尔可夫决策过程(MDP),通过可用测量和检测器反馈在线学习自适应策略,动态调节零均值高斯水印的协方差,无需系统先验知识。该框架设计了独特的奖励函数,实时平衡控制性能、能耗与检测置信度。针对线性系统,开发贝叶斯信念更新机制实现检测置信度的实时估计。在西门子Sinumerik 828D控制器数字孪生上,相较恒定方差基线,DynaMark实现水印能耗降低70%的同时保持原始轨迹不变,并维持平均检测延迟为一个采样间隔。物理步进电机实验平台验证结果,能快速触发警报,控制性能损失小,优于现有基准。
原文摘要 · Abstract (English)
Industry 4.0's highly networked Machine Tool Controllers (MTCs) are prime targets for replay attacks that use outdated sensor data to manipulate actuators. Dynamic watermarking can reveal such tampering, but current schemes assume linear-Gaussian dynamics and use constant watermark statistics, making them vulnerable to the time-varying, partly proprietary behavior of MTCs. We close this gap with DynaMark, a reinforcement learning framework that models dynamic watermarking as a Markov decision process (MDP). It learns an adaptive policy online that dynamically adapts the covariance of a zero-mean Gaussian watermark using available measurements and detector feedback, without needing system knowledge. DynaMark maximizes a unique reward function balancing control performance, energy consumption, and detection confidence dynamically. We develop a Bayesian belief updating mechanism for real-time detection confidence in linear systems. This approach, independent of specific system assumptions, underpins the MDP for systems with linear dynamics. On a Siemens Sinumerik 828D controller digital twin, DynaMark achieves a reduction in watermark energy by 70% while preserving the nominal trajectory, compared to constant variance baselines. It also maintains an average detection delay equivalent to one sampling interval. A physical stepper-motor testbed validates these findings, rapidly triggering alarms with less control performance decline and exceeding existing benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。