用测试反馈迭代优化，提升孟加拉语转代码准确率
Retriv at BLP-2025 Task 2: Test-Driven Feedback-Guided Framework for Bangla-to-Python Code Generation
- 基于测试驱动反馈，三次迭代优化生成代码
- 在共享任务中获第二名，单次通过率达93.4%
- 针对低资源语言设计，适合代码生成研究者参考
大型语言模型（LLMs）已推动自然语言到代码的自动化生成，但孟加拉语等低资源语言因指令-代码数据集和评估基准稀缺而进展缓慢。为此，IJCNLP-AACL 2025 年 BLP 工作坊设立了‘孟加拉语代码生成’共享任务。本文提出一种结合指令提示与测试驱动、反馈引导的迭代优化方法，采用微调后的 Qwen2.5-14B 模型，从孟加拉语指令生成代码，通过单元测试验证并根据失败反馈进行三轮迭代修正。该方法使团队 'Retriv' 在任务中取得第二名，Pass@1 得分为 0.934。分析揭示了孟加拉语理解与 Python 代码生成中的挑战，强调需为低资源语言设计针对性方法。实验脚本已公开，供社区使用。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have advanced the automated generation of code from natural language prompts. However, low-resource languages (LRLs) like Bangla remain underrepresented due to the limited availability of instruction-to-code datasets and evaluation benchmarks. To address this, the BLP Workshop at IJCNLP-AACL 2025 introduced a shared task on "Code Generation in Bangla". In this work, we propose a method that combines instruction prompting with a test-driven, feedback-guided iterative refinement process using a fine-tuned Qwen2.5-14B model. The model generates code from Bangla instructions, tests it against unit tests, and iteratively refines any failing outputs through three evaluation passes, using test feedback to guide each step. This approach helped our team "Retriv" to secure 2nd place in the shared task with a Pass@1 score of 0.934. The analysis highlights challenges in Bangla instruction understanding and Python code generation, emphasizing the need for targeted methods in LRLs. We made experimental scripts publicly available for the community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。