用小型语言模型+验证器闭环控制,实现边缘端自主调节
Closed-Loop Control with Rule-Aligned Small Language Models and Multi-Agent Self-Correction

- 用微调的小模型生成控制指令,通过数字孪生验证并迭代修正
- 热控模拟中动作对齐率达91.5%,平均推理延迟仅3.84秒
- 适合边缘部署的可重构工业自动控制,尤其看重低延迟场景
实现自主工业运行的关键在于能够根据自然语言需求生成或重构控制策略,而无需人工重设计。在此背景下,若结合具备工厂感知能力的验证器(如数字孪生)对候选动作进行执行前检查,AI代理即可可信地生成策略。但实际部署受限于推理延迟和计算开销:大型云端模型通常过慢、不透明或数据敏感,难以用于边缘闭环系统。本文研究是否可通过重新训练紧凑型小语言模型(SLM)实现控制推理,并嵌入由验证器引导的纠错循环。采用经组相对策略优化(GRPO)微调的Qwen2.5-1.5B模型,配合(i)动作生成代理,(ii)符号化/数字孪生式验证层,以及(iii)迭代重提示以引导输出向有效动作收敛的重提示代理。在随机热控仿真中(30次实验,每次500步),该框架实现了平均91.5%的动作对齐准确率(各案例间为86.3%–100%),平均推理延迟为3.84秒。在符号映射下,仍保持95%的范围内调控率,表明即使在令牌级一致性降低的情况下,仍具备稳健的物理调节能力。结果支持将SLM+验证器架构作为边缘可重构自主控制的实用路径。
原文摘要 · Abstract (English)
A key step toward autonomous industrial operation is the ability to create and reconfigure control policies from natural-language requirement specifications, with minimal or no manual redesign. In this setting, policy generation by AI agents can be a credible path when paired with a plant-aware validator (e.g., a digital twin) that can check generated candidate actions before execution. However, practical deployment is constrained by inference latency and compute footprint: large cloud-based models are often too slow, opaque, or data-sensitive for edge closed-loop use. This work investigates whether a compact Small Language Model (SLM) can be retrained for control reasoning and embedded in a validator-guided correction loop. We use a Qwen2.5-1.5B model aligned via Group Relative Policy Optimization (GRPO), combined with (i) an action agent, (ii) a symbolic/digital-twin-style validation layer, and (iii) a reprompting agent that iteratively steers outputs toward valid actions. In randomized thermal-control simulations (30 experiments with 500 steps each), the framework achieves 91.5% average action-alignment accuracy (86.3%--100% across cases) at 3.84\,s mean inference latency. Under symbolic re-mapping, it maintains a 95% in-range rate, indicating robust physical regulation despite reduced token-level agreement. These results support SLM+validator architectures as a practical path toward reconfigurable autonomous control at the edge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。