用ChatGPT写正式规格无效，学生越不信它反而做得越好
Do AI assistants help students write formal specifications? A study with ChatGPT and the B-Method
- 用ChatGPT辅助撰写B方法规格，但未提升正确性
- 学生对AI信任度低时，规格正确率反而更高
- 发现特定交互模式影响规格质量，适合形式化教学研究者
本文研究AI助手（以OpenAI的ChatGPT为例）在本科生形式化方法（FM）教学中的作用，采用B方法作为形式化规格技术。尽管已有研究证明AI在编程任务中有效，但尚无研究考察其对形式化规格的影响。我们探究了ChatGPT在编写B规格时是否具有优势，并分析学生对其输出的信任程度。结果表明，使用AI并未提高规格的正确性，且信任度低的学生表现更佳。此外，我们识别出一种与ChatGPT互动的行为模式，可能影响B规格的正确性。
原文摘要 · Abstract (English)
This paper investigates the role of AI assistants, specifically OpenAI's ChatGPT, in teaching formal methods (FM) to undergraduate students, using the B-method as a formal specification technique. While existing studies demonstrate the effectiveness of AI in coding tasks, no study reports on its impact on formal specifications. We examine whether ChatGPT provides an advantage when writing B-specifications and analyse student trust in its outputs. Our findings indicate that the AI does not help students to enhance the correctness of their specifications, with low trust correlating to better outcomes. Additionally, we identify a behavioural pattern with which to interact with ChatGPT which may influence the correctness of B-specifications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。