首次构建真实可执行的工业控制系统攻击评估框架，验证大模型能否持续造成物理破坏。
PLCBench: Can Autonomous LLM Agents Turn PLC Access into Sustained Physical Impact?
- 构建硬件在环测试平台，结合真实PLC与闭环仿真系统。
- 240次实验中31.3%成功实现持续物理影响，关键在过程观测质量。
- 揭示防御盲点：多数攻击停在读取阶段，写入后仍难维持目标。
工业控制系统依赖可编程逻辑控制器（PLC）将网络计算与物理控制连接。使用工具的大语言模型（LLM）代理构成新兴攻击威胁：自主代理能否将可网络访问的PLC转化为持续的负面物理影响？现有评估多集中于数字任务或单阶段测试，在工业系统中，仅完成软件入侵、合法写入或工具访问可能低估实际物理风险。我们提出PLCBENCH，据我们所知首个面向真实PLC的硬件在环（HIL）框架，用于刻画此类网络到物理的攻防能力及其边界。该框架整合厂商原生交互、商用PLC执行、闭环简化过程仿真及独立结果验证。确定性评估器基于固定规则对运行者、通信、PLC对象和过程记录分配六个隐藏诊断标志，以区分可用的PLC交互、过程关联操作与持续物理影响。我们在四种商用PLC上部署四个闭环工作负载，测试五种LLM家族共240个真实PLC实验回合。其中75个（31.3%）成功维持各自物理目标。分阶段结果显示，98个回合在有效原生读取前终止，62个到达过程关联写入但未持续达成目标。值得注意的是，更丰富的过程观测使过程关联写入后的条件目标达成率从44.2%提升至64.0%。这些测量定位了配置不当的PLC-过程部署中的失败点，并为未来防御评估指明干预节点。为支持可复现性，我们通过配套资源发布安全披露的PLCBENCH代码及纯软件复现流程。
原文摘要 · Abstract (English)
Industrial control systems (ICSs) rely on programmable logic controllers (PLCs) to connect networked computation with physical control. Tool-using large language model (LLM) agents represent an emerging attack threat: can an autonomous agent convert a network-reachable PLC into sustained adverse physical impact? However, existing evaluations focus on digital tasks or individual stages of PLC testing. In ICSs, evaluations that stop at software exploitation, an accepted write, or tool access may therefore mischaracterize physical risk. We present PLCBENCH, to our knowledge, the first real-PLC hardware-in-the-loop (HIL) framework for characterizing this cyber-to-physical capability and its boundaries. It combines vendor-native interaction, commercial PLC execution, closed-loop reduced-order process simulation, and independent outcome verification. A deterministic evaluator applies fixed rules to runner, communication, PLC-object, and process records to assign six hidden diagnostic flags, distinguishing usable PLC interaction, process-linked manipulation, and sustained physical impact. We instantiate PLCBENCH on four commercial PLCs crossed with four closed-loop workloads. Across five LLM families and 240 real-PLC episodes, 75 episodes (31.3%) sustain their respective physical objectives. Stagewise results show that 98 episodes stop before a valid native read, whereas 62 reach a process-linked write but do not sustain the final objective. Notably, richer process observation is associated with an increase in conditional objective attainment after a process-linked write from 44.2% to 64.0%. These measurements localize failure in configured PLC-process deployments and identify intervention points for future defense evaluation. To support reproducibility, we release the safely disclosable PLCBENCH code and a software-only reproduction pipeline through the accompanying artifact.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。