首个系统评估逻辑漏洞修复技术的框架，涵盖真实漏洞与LLM应用效果。
LogicEval: A Systematic Framework for Evaluating Automated Repair Techniques for Logical Vulnerabilities in Real-World Software
- 构建首个包含122个真实逻辑漏洞的数据集LogicDS
- LLM修复易受提示敏感性和上下文丢失影响，成功率受限
- 适合安全研究者与自动化修复工具开发者参考
软件中的逻辑漏洞源于程序逻辑缺陷而非内存安全问题，可能导致严重安全失效。现有自动化修复技术多聚焦于内存破坏漏洞，对逻辑漏洞修复能力有限，主要因缺乏对漏洞代码及其预期行为的语义理解。尽管大语言模型（LLMs）在代码理解和修复方面取得进展，但尚无系统框架评估其在逻辑漏洞上的表现。本文提出首个系统性评估框架LogicEval，用于分析传统与基于LLM的修复方法在真实世界逻辑漏洞上的有效性。为此，我们构建了首个数据集LogicDS，包含122个具有实际安全影响的逻辑漏洞。评估表明，编译与测试失败主要由提示敏感性、代码上下文丢失及补丁定位困难导致。
原文摘要 · Abstract (English)
Logical vulnerabilities in software stem from flaws in program logic rather than memory safety, which can lead to critical security failures. Although existing automated program repair techniques primarily focus on repairing memory corruption vulnerabilities, they struggle with logical vulnerabilities because of their limited semantic understanding of the vulnerable code and its expected behavior. On the other hand, recent successes of large language models (LLMs) in understanding and repairing code are promising. However, no framework currently exists to analyze the capabilities and limitations of such techniques for logical vulnerabilities. We aim to systematically evaluate both traditional and LLM based repair approaches for addressing real world logical vulnerabilities. To facilitate our assessment, we created the first ever dataset, LogicDS, comprising 122 logical vulnerabilities that reflect tangible security impact. We also developed a systematic framework, LogicEval, to evaluate patches for logical vulnerabilities. Evaluations suggest that compilation and testing failures are primarily driven by prompt sensitivity, loss of code context, and difficulty in patch localization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。