用大模型生成针对性测试，高效发现并修复代码漏洞。
Mutation-Guided LLM-based Test Generation at Meta
- 基于大模型生成特定缺陷的模拟变异体，精准定位问题。
- 在7个平台中生成571个隐私增强测试用例，73%被工程师采纳。
- 支持任意类型回归防护，适合关注安全与稳定性的研发团队。
本文介绍Meta的ACH系统，一种基于大模型的变异引导测试生成方法。ACH相比传统变异测试生成更少的变异体（共9,095个），但聚焦于当前未被检测到的、与特定问题相关的缺陷。通过生成可捕获这些缺陷的测试用例，实现对变异体的“杀死”，从而强化平台对抗回归的能力。以隐私问题为例，该系统在Meta的7个软件平台上应用，覆盖10,795个Android Kotlin类。此外，系统部署了基于大模型的等效变异体检测代理，精度达0.79，召回率达0.47（经简单预处理后提升至0.95和0.96）。在Messenger和WhatsApp的测试活动中，工程师采纳了73%的建议测试，其中36%被认为与隐私相关。结果表明，即使测试不直接针对特定问题，其带来的其他价值也使工程师普遍认可。
原文摘要 · Abstract (English)
This paper describes Meta's ACH system for mutation-guided LLM-based test generation. ACH generates relatively few mutants (aka simulated faults), compared to traditional mutation testing. Instead, it focuses on generating currently undetected faults that are specific to an issue of concern. From these currently uncaught faults, ACH generates tests that can catch them, thereby `killing' the mutants and consequently hardening the platform against regressions. We use privacy concerns to illustrate our approach, but ACH can harden code against {\em any} type of regression. In total, ACH was applied to 10,795 Android Kotlin classes in 7 software platforms deployed by Meta, from which it generated 9,095 mutants and 571 privacy-hardening test cases. ACH also deploys an LLM-based equivalent mutant detection agent that achieves a precision of 0.79 and a recall of 0.47 (rising to 0.95 and 0.96 with simple pre-processing). ACH was used by Messenger and WhatsApp test-a-thons where engineers accepted 73% of its tests, judging 36% to privacy relevant. We conclude that ACH hardens code against specific concerns and that, even when its tests do not directly tackle the specific concern, engineers find them useful for their other benefits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。