用自动化方法发现大模型文本转SQL的隐藏漏洞。
Beyond Static Rules: Automated Discovery of Latent Vulnerabilities in Text-to-SQL

- 构建动态漏洞知识库,自动设计针对性测试用例
- 在多个开源模型上发现大量未被识别的失败案例
- 发现的漏洞模式可跨模型复用,适合安全评估与模型优化
尽管大型语言模型(LLMs)在文本转SQL任务中取得显著进展,但其在真实环境中的部署受限于潜在的可靠性问题。识别这些隐性弱点对构建可信数据库接口至关重要,然而当前诊断方法依赖静态、专家定义的规则,缺乏系统性与自动化能力。为此,我们提出SAGE(系统化自动化引导探索)框架,可自主发现基于LLM的文本转SQL生成中的潜在故障模式。SAGE为给定样本生成漏洞假设,并参考持续演化的漏洞知识库(Vulnerability Codex)设计针对性扰动,迭代验证并记录潜在缺陷。在多个先进开源LLM上的实验表明,SAGE发现了大量此前未被识别的失败案例,揭示了现有模型的显著脆弱性。进一步分析显示,漏洞知识库具备强跨模型迁移能力,说明所发现模式代表普遍存在的结构性缺陷。最后,我们探索了SAGE在修复中的潜力:初步的轻量级微调在生成样本上取得显著改进,提示未来实现可靠性闭环的可行路径。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) have achieved remarkable success in Text-to-SQL tasks, their deployment in real-world environments is hindered by latent reliability issues. Identifying these latent weaknesses is critical for building trustworthy database interfaces, yet current diagnostic approaches rely heavily on static, expert-defined rules, which lack the capability for systematic and automated exploration. To bridge this gap, we propose SAGE (Systematic Automated Guided Exploration), a novel framework designed to autonomously uncover latent failure patterns in LLM-based Text-to-SQL generation. Specifically, SAGE generates vulnerability hypotheses for given samples and references a continuously evolving Vulnerability Codex to design targeted perturbations, thereby iteratively verifying and documenting potential defects. Extensive experiments on state-of-the-art open-source LLMs demonstrate that SAGE uncovers a substantial number of failure cases, highlighting the significant fragility of current models. Furthermore, our analysis reveals that the Vulnerability Codex exhibits strong cross-model transferability, indicating that the discovered patterns represent generalized structural weaknesses. Finally, we explore SAGE's potential for remediation. Although preliminary, lightweight fine-tuning on the generated samples yields promising improvements, suggesting a scalable pathway for closing the reliability loop in future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。