企业合规助手通过智能路由提升任务效率,响应更快且更准确。
Compliance Brain Assistant: Conversational Agentic AI for Assisting Compliance Tasks in Enterprise Environments
- 设计智能路由机制,自动选择快速检索或复杂任务处理模式。
- 关键词匹配率83.7%、判断通过率82.0%,远超普通大模型的41.7%和20.0%。
- 适合需要高效处理合规任务的企业用户,兼顾速度与准确性。
本文提出合规大脑助手(CBA),一种面向企业环境的对话式智能体AI助手,旨在提升人员日常合规任务的效率。为平衡响应质量与延迟,设计了用户查询路由器,可智能选择:(i) 快速通道模式——仅需从知识库中检索相关上下文以处理简单请求;(ii) 全智能体模式——处理需复合操作、工具调用以跨文档主动发现信息或调用其他API/模型的复杂请求。例如,根据用户问题定位特定实体,再利用其信息调用外部API生成丰富响应。实验对比CBA与通用大模型在真实隐私/合规场景下的表现,结果显示CBA在平均关键词匹配率(83.7% vs. 41.7%)和LLM判别通过率(82.0% vs. 20.0%)上显著领先。进一步对比路由设计与仅快速通道或仅全智能体模式,发现前者在保持近似运行时间的同时,实现更高平均匹配率与通过率,验证了路由机制在质量与效率间的良好权衡。
原文摘要 · Abstract (English)
This paper presents Compliance Brain Assistant (CBA), a conversational, agentic AI assistant designed to boost the efficiency of daily compliance tasks for personnel in enterprise environments. To strike a good balance between response quality and latency, we design a user query router that can intelligently choose between (i) FastTrack mode: to handle simple requests that only need additional relevant context retrieved from knowledge corpora; and (ii) FullAgentic mode: to handle complicated requests that need composite actions and tool invocations to proactively discover context across various compliance artifacts, and/or involving other APIs/models for accommodating requests. A typical example would be to start with a user query, use its description to find a specific entity and then use the entity's information to query other APIs for curating and enriching the final AI response. Our experimental evaluations compared CBA against an out-of-the-box LLM on various real-world privacy/compliance-related queries targeting various personas. We found that CBA substantially improved upon the vanilla LLM's performance on metrics such as average keyword match rate (83.7% vs. 41.7%) and LLM-judge pass rate (82.0% vs. 20.0%). We also compared metrics for the full routing-based design against the `fast-track only` and `full-agentic` modes and found that it had a better average match-rate and pass-rate while keeping the run-time approximately the same. This finding validated our hypothesis that the routing mechanism leads to a good trade-off between the two worlds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。