用大模型自动检查合规性,精准识别企业是否符合监管要求。
CTRAG: An In-Context Retrieval-based Framework for Automated Compliance Checking using LLMs

- 基于检索增强生成,从法规中提取问题并比对公司文档
- 在真实场景中实现78%的F1分数和85%召回率
- 适合审计、风控等需要高效合规核查的机构使用
信任是现代监管生态的基础,合规检查在建立这种信任中起着关键作用。在高度受控环境中运营的企业需要确保其符合金融报告、数据隐私和网络安全等领域的行业规范。然而,人工合规测试通常耗时且易出错,尤其是在依赖第三方服务(如云服务商)间接合规的情况下。本文提出CTRAG,一种新型基于上下文检索的生成式合规检查框架。该框架采用自适应分块、动态检索配置和上下文学习策略,提升评估精度与相关性。通过从监管文本中提取控制问题,并与非结构化公司文档交叉比对,即使在间接合规场景下也能实现高精度验证。实证评估显示,最终部署配置下,CTRAG达到78%的F1分数和85%的召回率,显著减少漏检,降低人工审查负担。我们在一家四大会计师事务所开发并部署了原型系统,应用于真实案例,并与人工报告对比验证。结果表明,CTRAG有望简化合规流程、降低风险,增强复杂高风险环境中的监管可信度。
原文摘要 · Abstract (English)
Trust is fundamental in modern regulatory ecosystems, and compliance checking plays a critical role in fostering that trust. Regulatory compliance verification is essential for businesses operating in highly controlled environments, as it ensures alignment with sector-specific guidelines across domains such as financial reporting, data privacy, and cybersecurity. Manual compliance testing, however, is often time-intensive and prone to inconsistencies, particularly when compliance depends indirectly on third-party services such as cloud providers, where vendors rely on external providers to meet regulatory standards. In this paper, we present CTRAG, a novel Retrieval-Augmented Generation (RAG) pipeline designed for automated compliance checking. CTRAG employs advanced strategies, including adaptive chunking, dynamic retrieval configurations, and in-context learning, to improve the precision and relevance of compliance assessments. By extracting control questions from regulatory texts and cross-referencing them with unstructured company documentation, CTRAG achieves highly accurate, document-informed compliance verification, even in cases of indirect compliance through third-party services. Empirical evaluations demonstrate significant improvements, with CTRAG achieving an F1-score of 78% and a recall of 85% in the final deployed configuration, ensuring minimal missed non-compliance cases while reducing manual reviewer effort in a real-world deployment. To validate CTRAG value, we developed and deployed a POC within a Big Four professional services firm, applying it to real-world cases and cross-checking results against manual compliance reports. These findings highlight CTRAG potential to streamline compliance workflows, mitigate risks, and enhance regulatory trust in complex, high-stakes environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。