通过错误驱动优化提示，让小模型在本地算准数学题
Error-Driven Prompt Optimization for Arithmetic Reasoning
- 根据错误预测聚类，迭代优化提示规则
- 小模型准确率提升至70.8%，超越GPT-3.5 Turbo
- 适合需要隐私保护的金融医疗等工业场景
人工智能进展推动了在金融、医疗等受监管领域支持分析师的工业级智能体发展。此类系统需在结构化数据上执行精确算术运算，同时确保敏感信息不离开本地安全环境。本文提出一种面向算术推理的错误驱动优化框架，增强代码生成代理（CGA），应用于本地小型语言模型（SLMs）。对领先的小模型Qwen3 4B的系统评估显示，基础模型在算术任务中存在根本性局限，而本文提出的错误驱动方法通过聚类错误预测并迭代优化提示规则，显著提升性能，使准确率达70.8%。结果表明，无需昂贵微调，仅靠系统性错误驱动提示优化，即可实现可靠、可解释且可部署的工业级AI助手，使小模型在隐私合规前提下超越大型模型（GPT-3.5 Turbo）。
原文摘要 · Abstract (English)
Recent advancements in artificial intelligence have sparked interest in industrial agents capable of supporting analysts in regulated sectors, such as finance and healthcare, within tabular data workflows. A key capability for such systems is performing accurate arithmetic operations on structured data while ensuring sensitive information never leaves secure, on-premises environments. Here, we introduce an error-driven optimization framework for arithmetic reasoning that enhances a Code Generation Agent (CGA), specifically applied to on-premises small language models (SLMs). Through a systematic evaluation of a leading SLM (Qwen3 4B), we find that while the base model exhibits fundamental limitations in arithmetic tasks, our proposed error-driven method, which clusters erroneous predictions to refine prompt-rules iteratively, dramatically improves performance, elevating the model's accuracy to 70.8\%. Our results suggest that developing reliable, interpretable, and industrially deployable AI assistants can be achieved not only through costly fine-tuning but also via systematic, error-driven prompt optimization, enabling small models to surpass larger language models (GPT-3.5 Turbo) in a privacy-compliant manner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。