用代码预测测试能否通过，提速自动测试生成
RM -RF: Reward Model for Run-Free Unit Test Evaluation
- 仅凭源码和测试代码预测编译运行、覆盖率、杀灭率
- 多语言数据集上三任务平均F1达0.69，接近真实执行效果
- 适合大规模测试生成与强化学习优化，降低算力开销
我们提出轻量级奖励模型RM-RF，用于无需实际运行的自动化单元测试评估。不依赖反复编译与执行候选测试，仅从源码和测试代码即可预测三个执行相关信号：（1）增强测试套件是否能成功编译运行，（2）生成测试是否提升代码覆盖率，（3）生成测试是否提高变异杀灭率。为训练和评估RM-RF，我们构建了涵盖Java、Python、Go的多语言数据集，包含焦点文件、测试文件及由执行管道标注的候选测试添加项，并公开配套数据集与评估方法。测试了多种模型架构与微调策略（零样本、全量微调、基于LoRA的参数高效微调），在三个目标上平均F1达0.69。相比传统编译-运行工具，RM-RF显著降低延迟与基础设施成本，同时保持竞争性预测精度，可实现大规模测试生成与基于强化学习的代码优化的快速、可扩展反馈。
原文摘要 · Abstract (English)
We present RM-RF, a lightweight reward model for run-free evaluation of automatically generated unit tests. Instead of repeatedly compiling and executing candidate tests, RM-RF predicts - from source and test code alone - three execution-derived signals: (1) whether the augmented test suite compiles and runs successfully, (2) whether the generated test cases increase code coverage, and (3) whether the generated test cases improve the mutation kill rate. To train and evaluate RM-RF we assemble a multilingual dataset (Java, Python, Go) of focal files, test files, and candidate test additions labeled by an execution-based pipeline, and we release an associated dataset and methodology for comparative evaluation. We tested multiple model families and tuning regimes (zero-shot, full fine-tuning, and PEFT via LoRA), achieving an average F1 of 0.69 across the three targets. Compared to conventional compile-and-run instruments, RM-RF provides substantially lower latency and infrastructure cost while delivering competitive predictive fidelity, enabling fast, scalable feedback for large-scale test generation and RL-based code optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。