用伪代码和真实代码双执行提升测试输出预测准确率
DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocode
- 结合伪代码推理与真实代码执行，双重验证预测结果
- 在LiveCodeBench上将通过率提升13.6个百分点
- 适合需要高可靠测试生成的开发者和自动化测试研究者
本文针对测试输出预测这一测试用例生成中的关键挑战，提出改进大模型预测可靠性的方法。现有方法先生成代码以锚定预测结果，但代码中微小错误易导致执行失败。为此，本文引入基于大模型的伪代码执行，通过大模型推理模拟伪代码运行，具备更强容错性。进一步提出DuET双执行框架，融合代码直接执行与伪代码推理，采用功能多数投票机制。分析表明二者互补：代码执行克服伪代码幻觉，伪代码推理缓解代码错误影响。在LiveCodeBench数据集上，DuET实现当前最优性能，Pass@1提升13.6个百分点。
原文摘要 · Abstract (English)
This work addresses test output prediction, a key challenge in test case generation. To improve the reliability of predicted outputs by LLMs, prior approaches generate code first to ground predictions. One grounding strategy is direct execution of generated code, but even minor errors can cause failures. To address this, we introduce LLM-based pseudocode execution, which grounds prediction on more error-resilient pseudocode and simulates execution via LLM reasoning. We further propose DuET, a dual-execution framework that combines both approaches by functional majority voting. Our analysis shows the two approaches are complementary in overcoming the limitations of direct execution suffering from code errors, and pseudocode reasoning from hallucination. On LiveCodeBench, DuET achieves the state-of-the-art performance, improving Pass@1 by 13.6 pp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。