用多智能体进化搜索生成更鲁棒的单元测试边角用例
A Multi-Agent Framework for Stateful Inference-Time Search
- 引入持续状态和进化机制,让智能体逐步优化测试用例
- 在HumanEval等基准上覆盖率提升显著,优于单步无状态方法
- 无需微调,适合生成复杂代码的高覆盖边角测试
近期研究探索了代理式推理时技术以实现结构化、多步推理。然而,无状态推理在多步任务中常因缺乏持久状态而表现不佳。此外,特定任务微调或指令微调虽能生成表面代码,但在需要深层推理和长程依赖的任务中仍显脆弱。为此,我们提出有状态多智能体演化搜索——一种无需训练的框架,通过(i)推理时持久状态、(ii)对抗性变异、(iii)演化保留机制突破传统无状态方法局限。我们在自动化单元测试生成中验证其有效性,通过演化搜索过程生成鲁棒边角用例:专业化智能体依次提出、变异并评分候选用例,控制器维持跨代持久状态,演化保留确保全可能用例的多样性与探索性。该框架可生成覆盖广泛未见代码库的高覆盖边角用例。实验表明,在HumanEval、TestGenEvalMini等主流单元测试基准上,使用Llama、Gemma、GPT三类大模型,本框架相较无状态单步基线取得显著覆盖率提升,证明结合持久推理状态与演化搜索能显著改善单元测试生成效果。
原文摘要 · Abstract (English)
Recent work explores agentic inference-time techniques to perform structured, multi-step reasoning. However, stateless inference often struggles on multi-step tasks due to the absence of persistent state. Moreover, task-specific fine-tuning or instruction-tuning often achieve surface-level code generation but remain brittle on tasks requiring deeper reasoning and long-horizon dependencies. To address these limitations, we propose stateful multi-agent evolutionary search, a training-free framework that departs from prior stateless approaches by combining (i) persistent inference-time state, (ii) adversarial mutation, and (iii) evolutionary preservation. We demonstrate its effectiveness in automated unit test generation through the generation of edge cases. We generate robust edge cases using an evolutionary search process, where specialized agents sequentially propose, mutate, and score candidates. A controller maintains persistent state across generations, while evolutionary preservation ensures diversity and exploration across all possible cases. This yields a generalist agent capable of discovering robust, high-coverage edge cases across unseen codebases. Experiments show our stateful multi-agent inference framework achieves substantial gains in coverage over stateless single-step baselines, evaluated on prevalent unit-testing benchmarks such as HumanEval and TestGenEvalMini and using three diverse LLM families - Llama, Gemma, and GPT. These results indicate that combining persistent inference-time state with evolutionary search materially improves unit-test generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。