CLAP通过闭环流程提升领域模型后训练效果,解决数据噪声与发布风险问题。
CLAP: Closed-Loop Training, Evaluation, and Release Control for Domain Agent Post-training

- 构建闭环系统,将业务数据转化为结构化训练与评估样本
- 5批制造场景中平均得分提升0.0098,但仅3批有效,部分出现性能下降
- 强调应用链重放与RAG必要性,适合关注模型发布安全的工业用户
领域代理常面临业务数据噪声、后训练收益不确定、离线与应用不匹配及适配器发布风险。本文提出CLAP(闭环代理后训练),将业务数据转化为结构化SFT样本、决策偏好样本、保留集、风险诊断与发布门控记录。CLAP融合数据验证、目标/证据归一化、奖励/KL诊断、离线门控与应用链重放,判断适配器是否适合目标应用链。在五个匿名制造场景批次中,基于QLoRA的LoRA-SFT带来微小平均增益:总体得分提升0.0098,通过率提高0.0240,证据准确率提升0.0280,幻觉与错误事实减少;但仅3批改善,部分批次退化,GRPO暴露高KL风险。应用链重放进一步显示,RAG对事实提取至关重要;在相同3B主干与100次重放下,面向RAG的LoRA-SFT适配器优于基线+RAG,提升价值、核心字段与答案-证据文档/页匹配度,但增加延迟。结果表明,应通过整合的数据-训练-评估-发布闭环管理领域代理后训练,而非依赖训练完成或单一离线指标。
原文摘要 · Abstract (English)
Domain agents often face noisy business data, uncertain post-training gains, offline/application mismatch, and adapter-release risk. This paper presents CLAP (Closed-Loop Agent Post-training), a closed-loop method that converts business data into structured SFT samples, decision-preference samples, holdout sets, risk diagnostics, and release-gate records. CLAP combines data validation, target/evidence normalization, reward/KL diagnosis, offline gates, and application-chain replay to decide whether an adapter is suitable for the target application chain. On five anonymized manufacturing-scenario batches, QLoRA-style LoRA-SFT yields modest average gains: overall score increases by 0.0098, pass rate by 0.0240, and evidence accuracy by 0.0280, while hallucination and wrong facts decrease. Yet only 3 of 5 batches improve, some batches regress, and GRPO exposes high KL risks. Application-chain replay further shows that RAG is necessary for factual extraction; under the same 3B backbone and 100 replay cases, an application-RAG-oriented LoRA-SFT adapter improves value, core fields, and answer-evidence doc/page matching over base+RAG, but increases latency. These results support managing domain-agent post-training through an integrated data-training-evaluation-release loop rather than relying on training completion or a single offline score.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。