arXiv:2608.28147cs.SEcs.AI2026-08

明确提示验证频率能显著提升工程智能体的验证行为与成功率。

Post-Edit Re-Verification in Simulator-Backed Engineering Agents: A Controlled Comparison of Verification-Cadence Guidance

  • 对比有无显式验证节奏提示,考察模型执行修改后验证的行为差异。
  • 有提示时94%任务完成再验证,失败率降低至26%,最终成功率达95%。
  • 适用于需严格遵循流程的自动化工程设计系统研发与评估。

与外部仿真器交互的工程智能体在修改设计后,需重新获取证据以验证新状态。本文研究在保持验证相关状态不变的前提下,保留或省略显式验证节奏指导,对修改后重验证行为的影响。使用DWSIM作为仿真后端,进行连续阀压调节,评估了五个阿里/通义模型在八组合成案例中的表现;每组模型-案例-条件组合通过实时API调用执行三次,每种条件共120次评估。结果显示,在保留节奏指导(CG)条件下,94/120次任务完成再验证,仅26/120次出现节奏违规,最终成功达95/120次;而省略指导(CO)条件下,仅32/120次再验证,87/120次节奏违规,最终成功35/120次。qwen3.5-35b-a3b在两种条件下均表现极差(CG仅1/24次再验证,无成功)。结果表明,在此受限协议中,显式验证节奏指导能有效提升再验证率、减少违规、提高成功率,支持将验证节奏作为交互协议的明确组成部分。

原文摘要 · Abstract (English)

Engineering agents that interact with external simulators may need to coordinate design modification with reacquisition of engineering evidence for the modified state. We ask whether first post-edit re-verification changes when explicit verification-cadence guidance is retained versus omitted while verification-relevant state/facts are held constant. Cadence-Guided (CG) retained an instruction to request a new simulation after a substantive modification, whereas Cadence-Omitted (CO) removed that instruction; neither condition used a hard gate. The study therefore measures instruction-conditioned post-edit verification-policy adherence rather than spontaneous recognition that prior evidence has become stale. Using DWSIM as the simulator backend and continuous valve-pressure adjustment, five Alibaba/Qwen models were evaluated on eight synthetic cases; each model-case-condition combination was executed three times via live API calls, yielding 120 evaluation slots per condition. Re-verification was observed in 94/120 CG slots versus 32/120 CO slots; cadence violations occurred in 26/120 versus 87/120; and bounded final success was reached in 95/120 versus 35/120. qwen3.5-35b-a3b showed minimal re-verification (1/24 in CG and 0/24 in CO) and no final success in either condition. Within this bounded protocol, explicit post-edit verification-cadence guidance was associated with more re-verification, fewer cadence violations, and more frequent bounded final success, supporting the treatment of verification cadence as an explicit interaction-protocol component.

工程智能体验证节奏仿真系统自动化设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。