用大模型分析代码最坏情况约束,发现性能瓶颈和漏洞
Worst-Case Symbolic Constraints Analysis and Generalisation with Large Language Models
- 分步推理:先在小输入上用工具生成约束,再让大模型推广到大输入
- 新模型WARP-1.0-3B在测试中超越更大规模基线模型
- 适合做软件安全检测和程序优化的研究者使用
大型语言模型(LLMs)在代码生成、补全和修复任务中表现优异,但在复杂符号推理方面仍显不足。本文提出最坏情况符号约束分析任务,旨在推导出导致程序最差执行的符号约束,这些约束可求解以获取暴露性能瓶颈或拒绝服务漏洞的输入。实验表明,即使是最先进的模型(如GPT-5)在直接应用时也表现不佳。为此,我们提出WARP——一种创新的神经符号方法:首先利用现有程序分析工具在较小具体输入上计算最坏情况约束,再通过大模型将其推广至更大输入规模。WARP包含三项核心技术:(1) 基于大模型的增量式最坏情况推理策略;(2) 与求解器对齐的神经符号框架,结合强化学习与SMT(可满足性模理论)求解;(3) 一个精心构建的符号约束数据集。实验显示,WARP在最坏情况约束推理任务中持续提升性能。基于该数据集,我们使用强化学习微调出模型WARP-1.0-3B,其显著优于同规模及更大规模基线模型。结果表明,增量约束推理能增强大模型的符号推理能力,并揭示神经学习与形式化方法深度融合在严谨程序分析中的潜力。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated strong performance on coding tasks such as generation, completion and repair, but their ability to handle complex symbolic reasoning over code still remains underexplored. We introduce the task of worst-case symbolic constraints analysis, which requires inferring the symbolic constraints that characterise worst-case program executions; these constraints can be solved to obtain inputs that expose performance bottlenecks or denial-of-service vulnerabilities in software systems. We show that even state-of-the-art LLMs (e.g., GPT-5) struggle when applied directly on this task. To address this challenge, we propose WARP, an innovative neurosymbolic approach that computes worst-case constraints on smaller concrete input sizes using existing program analysis tools, and then leverages LLMs to generalise these constraints to larger input sizes. Concretely, WARP comprises: (1) an incremental strategy for LLM-based worst-case reasoning, (2) a solver-aligned neurosymbolic framework that integrates reinforcement learning with SMT (Satisfiability Modulo Theories) solving, and (3) a curated dataset of symbolic constraints. Experimental results show that WARP consistently improves performance on worst-case constraint reasoning. Leveraging the curated constraint dataset, we use reinforcement learning to fine-tune a model, WARP-1.0-3B, which significantly outperforms size-matched and even larger baselines. These results demonstrate that incremental constraint reasoning enhances LLMs' ability to handle symbolic reasoning and highlight the potential for deeper integration between neural learning and formal methods in rigorous program analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。