用测试驱动开发提升大模型生成的表格代码准确性与可靠性。
Leveraging Test Driven Development with Large Language Models for Reliable and Verifiable Spreadsheet Code Generation: A Research Framework
- 将测试先行理念融入大模型提示词,引导生成可验证的代码。
- 通过测试约束减少幻觉和逻辑错误,提升结果可信度。
- 适合无编程基础的用户在金融、科研等高风险场景使用。
大型语言模型(如ChatGPT)被广泛用于生成传统软件代码和电子表格逻辑。尽管生成能力强大,但常出现幻觉、细微逻辑不一致和语法错误,尤其在财务建模和科学计算等高风险领域,准确性和可靠性至关重要。本文提出一种结构化研究框架,将经过验证的软件工程实践——测试驱动开发(TDD)与大模型生成相结合,以提升生成输出的正确性、可靠性和用户信心。我们假设‘先写测试’的方法能提供技术约束和认知支持,引导大模型输出更准确、可验证且易懂的解决方案。该框架适用于从电子表格公式生成到Python脚本乃至强类型语言(如Rust)等多种编程场景,包含明确的实验设计、分组设置、评估指标及基于TDD的提示示例。通过强调测试驱动思维,旨在提升计算思维、提示工程能力和用户参与度,尤其帮助缺乏正式编程训练的电子表格用户避免因逻辑错误带来的严重后果。本文诚邀合作,进一步优化并实证评估该方法,最终推动负责任、可靠的大型语言模型在教育与职业开发中的应用。
原文摘要 · Abstract (English)
Large Language Models (LLMs), such as ChatGPT, are increasingly leveraged for generating both traditional software code and spreadsheet logic. Despite their impressive generative capabilities, these models frequently exhibit critical issues such as hallucinations, subtle logical inconsistencies, and syntactic errors, risks particularly acute in high stakes domains like financial modelling and scientific computations, where accuracy and reliability are paramount. This position paper proposes a structured research framework that integrates the proven software engineering practice of Test-Driven Development (TDD) with Large Language Model (LLM) driven generation to enhance the correctness of, reliability of, and user confidence in generated outputs. We hypothesise that a "test first" methodology provides both technical constraints and cognitive scaffolding, guiding LLM outputs towards more accurate, verifiable, and comprehensible solutions. Our framework, applicable across diverse programming contexts, from spreadsheet formula generation to scripting languages such as Python and strongly typed languages like Rust, includes an explicitly outlined experimental design with clearly defined participant groups, evaluation metrics, and illustrative TDD based prompting examples. By emphasising test driven thinking, we aim to improve computational thinking, prompt engineering skills, and user engagement, particularly benefiting spreadsheet users who often lack formal programming training yet face serious consequences from logical errors. We invite collaboration to refine and empirically evaluate this approach, ultimately aiming to establish responsible and reliable LLM integration in both educational and professional development practices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。