让大模型生成代码更可靠，自动验证意图与结果一致性。
Talk Less, Verify More: Improving LLM Assistants with Semantic Checks and Execution Feedback
- 用反向翻译+语义匹配验证代码是否符合用户意图
- 通过执行反馈引导代码迭代优化，错误率下降37%
- 适合企业级智能助手开发，提升决策可信度
随着大语言模型助手在企业工作流中日益普及，其生成结果的准确性、语义一致性及可执行性至关重要。然而，现有对话式商业分析系统普遍缺乏内置验证机制，导致用户需手动核对可能出错的结果。本文提出两种互补的验证技术：Q* 通过代码与用户意图之间的反向翻译和语义匹配进行验证；Feedback+ 则引入执行反馈以指导代码优化。二者嵌入生成-判别框架中，将验证责任从用户转移至系统。在 Spider、Bird 和 GSM8K 三个基准数据集上的评估表明，Q* 与 Feedback+ 均能有效降低错误率并缩短任务完成时间。研究还发现反向翻译是主要瓶颈，提示未来改进方向。本工作为构建更可靠的、面向企业级的生成式 AI 系统提供了设计框架，支持可信决策支持。
原文摘要 · Abstract (English)
As large language model (LLM) assistants become increasingly integrated into enterprise workflows, their ability to generate accurate, semantically aligned, and executable outputs is critical. However, current conversational business analytics (CBA) systems often lack built-in verification mechanisms, leaving users to manually validate potentially flawed results. This paper introduces two complementary verification techniques: Q*, which performs reverse translation and semantic matching between code and user intent, and Feedback+, which incorporates execution feedback to guide code refinement. Embedded within a generator-discriminator framework, these mechanisms shift validation responsibilities from users to the system. Evaluations on three benchmark datasets, Spider, Bird, and GSM8K, demonstrate that both Q* and Feedback+ reduce error rates and task completion time. The study also identifies reverse translation as a key bottleneck, highlighting opportunities for future improvement. Overall, this work contributes a design-oriented framework for building more reliable, enterprise-grade GenAI systems capable of trustworthy decision support.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。