arXiv:2605.27981cs.AI2026-05

用自然语言说明书生成能暴露算法瓶颈的测试用例。

STAB: Specification-driven Testing for Algorithmic Bottlenecks

论文配图:STAB: Specification-driven Testing for Algorithmic Bottlenecks
图 1 · 摘自论文原文
  • 从问题说明出发,通过约束饱和和对抗结构注入生成测试用例。
  • 在CodeContests上使暴露瓶颈的测试用例率提升至73.45%(开源大模型)。
  • 适合算法优化、自动测试和大模型辅助编程研究者使用。

评估算法代码效率需要能暴露运行时瓶颈的测试用例。以往方法通过增大输入规模或生成特定代码的慢速输入来生成效率测试用例,但未针对驱动算法最坏情况的结构性输入条件。我们提出STAB,一种仅依赖自然语言问题说明的规范驱动测试生成流程。STAB将任务分解为约束边界最大化与对抗结构注入:(i) 约束饱和器通过规则饱和与CP-SAT优化提取约束并求解大可接受规模分配;(ii) 对抗场景注入器通过关键词匹配与KNN从精选场景库中检索实现级对抗构造原则。STAB将问题说明、解析边界与提取构造原则编码为结构化生成规范,由LLM合成Python测试用例生成器。在CodeContests上,STAB将开源LLM生成的测试用例暴露瓶颈率从50.43%提升至73.45%,闭源LLM从57.45%提升至71.85%,对Python、Java、C++均有稳定增益。

原文摘要 · Abstract (English)

Evaluating the efficiency of algorithmic code requires test cases that expose runtime bottlenecks. Previous methods generate efficiency test cases either by increasing input size or by generating code-specific inputs that make the given implementation run slowly. Consequently, they do not address the structural input conditions that drive the algorithmic worst case. We introduce STAB, a specification-driven pipeline that generates test cases that expose algorithmic bottlenecks from a natural-language problem specification alone. STAB separates the task into constraint-bound maximization and adversarial structure injection. (i) The constraint saturator extracts constraints and resolves large admissible size assignments using rule-based saturation and CP-SAT optimization over related variables. (ii) The adversarial scenario injector retrieves implementation-level adversarial construction principles from a curated scenario catalog using keyword matching and K-nearest neighbors (KNN). STAB encodes the problem specification, resolved boundary, and retrieved construction principles into a structured generation specification, from which the LLM synthesizes a Python test case generator. On CodeContests, STAB raises the rate of generated test cases that expose algorithmic bottlenecks from 50.43% to 73.45% on average across open-source LLMs and from 57.45% to 71.85% on average across closed-source LLMs, with consistent gains across Python, Java, and C++. Our code is available at https://github.com/suhanmen/STAB.

算法测试LLM应用自动化测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。