用图结构建模电商申诉审核中的纠错逻辑,提升AI判断准确率。
Reviewing the Reviewer: Graph-Enhanced LLMs for E-commerce Appeal Adjudication
- 构建证据-动作-因素-决策框架,用可验证操作约束推理过程。
- 通过历史案例构建知识图谱,实现95.8%的离线准确率。
- 能精准识别缺什么信息,自动发起针对性补充请求。
层级化审核流程中,二级审核员(Checker)修正一级审核员(Maker)的判断,产生有价值的纠错信号,反映初始判断失败原因。但学习这些信号受信息不对称限制:纠正往往依赖一级审核员或系统无法获取的验证操作。本文提出显式动作建模作为推理约束,使判断基于可验证操作而非自由文本生成。设计证据-动作-因素-决策(EAFD)最小表示形式,通过操作接地防止幻觉,并支持通过显式冲突建模学习纠错信号。在此基础上,构建冲突感知图推理框架:(1)从历史案例构建体现Maker-Checker分歧的EAFD图;(2)聚合为可检索的知识库;(3)对新案例进行自上而下的演绎推理,复现已验证的解决路径。独特能力是‘请求更多信息’(RMI)结果:当证据不足时,系统精准识别未执行的验证操作,并生成针对性信息请求。在大规模电商卖家申诉审核中评估,纯LLM基线与专家一致率为70.8%,引入动作建模与RMI后提升至87.5%;结合检索式知识图谱达到最优离线性能95.8%。上线部署后,生产环境仍保持96.3%的一致率,证明其实际有效性。
原文摘要 · Abstract (English)
Hierarchical review workflows, where a second-tier reviewer (Checker) corrects first-tier (Maker) decisions, generate valuable correction signals that encode why initial judgments failed. However, learning from these signals is hindered by information asymmetry: corrections often depend on verification actions unavailable to Makers or automated systems. We address this challenge by introducing explicit action modeling as an inferential constraint that grounds reasoning in verifiable operations rather than unconstrained text generation. We propose the Evidence-Action-Factor-Decision (EAFD) schema, a minimal representation for adjudication reasoning that prevents hallucination through operational grounding and enables learning from correction signals via explicit conflict modeling. Building on this schema, we develop a conflict-aware graph reasoning framework that: (1) constructs EAFD graphs from historical cases capturing Maker-Checker disagreements, (2) aggregates them into a retrievable knowledge base, and (3) performs top-down deductive reasoning for new cases by projecting validated resolution paths from precedents. A distinctive capability is the Request More Information (RMI) outcome: when evidence is insufficient, the system identifies precisely which verification actions remain unexecuted and generates targeted information requests. We evaluate the framework in large-scale e-commerce seller appeal adjudication. While a standard LLM-only baseline achieves only 70.8% alignment with human experts, incorporating action modeling with RMI improves alignment to 87.5%. Augmenting this with the retrieval-based knowledge graph yields the best offline performance of 95.8%. Following online deployment, the framework maintains robust performance, achieving a 96.3% alignment rate in production, demonstrating its real-world effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。