用形式验证反馈自动优化提示词,实现无需微调的机器人安全规划
LAD-VF: LLM-Automatic Differentiation Enables Fine-Tuning-Free Robot Planning from Formal Methods Feedback
- 通过形式验证反馈引导提示词迭代优化,不修改模型参数
- 机器人导航与操作任务中规范符合率从60%提升至90%以上
- 适合需高可靠性、可解释性的工业级AI控制场景
大型语言模型(LLMs)可将自然语言指令转化为机器人、自动驾驶等领域的可执行行动规划。然而,在物理世界部署基于LLM的规划时,必须严格遵守安全与法规约束,而现有模型常因幻觉或对齐不足导致违规。传统数据驱动对齐方法(如直接偏好优化DPO)需昂贵的人工标注,近期的形式反馈方法仍依赖资源密集的微调。本文提出LAD-VF,一种无需微调的框架,利用形式验证反馈实现自动化提示工程。通过引入与LLM-AutoDiff结合的形式验证反馈文本损失,LAD-VF迭代优化提示词而非模型参数。该方法带来三大优势:(i) 无需微调即可实现可扩展适应;(ii) 兼容模块化LLM架构;(iii) 通过可审计提示词实现可解释性优化。在机器人导航与操作任务中的实验表明,LAD-VF显著提升规范符合率,成功率达90%以上。本方法为可信、形式验证的LLM驱动控制系统提供了可扩展且可解释的路径。
原文摘要 · Abstract (English)
Large language models (LLMs) can translate natural language instructions into executable action plans for robotics, autonomous driving, and other domains. Yet, deploying LLM-driven planning in the physical world demands strict adherence to safety and regulatory constraints, which current models often violate due to hallucination or weak alignment. Traditional data-driven alignment methods, such as Direct Preference Optimization (DPO), require costly human labeling, while recent formal-feedback approaches still depend on resource-intensive fine-tuning. In this paper, we propose LAD-VF, a fine-tuning-free framework that leverages formal verification feedback for automated prompt engineering. By introducing a formal-verification-informed text loss integrated with LLM-AutoDiff, LAD-VF iteratively refines prompts rather than model parameters. This yields three key benefits: (i) scalable adaptation without fine-tuning; (ii) compatibility with modular LLM architectures; and (iii) interpretable refinement via auditable prompts. Experiments in robot navigation and manipulation tasks demonstrate that LAD-VF substantially enhances specification compliance, improving success rates from 60% to over 90%. Our method thus presents a scalable and interpretable pathway toward trustworthy, formally-verified LLM-driven control systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。