为反欺诈与反洗钱场景定制了高效合规的LLM部署方案。
Rethinking LLMOps for Fraud and AML: Building a Compliance-Grade LLM Serving Stack
- 针对合规提示特点优化缓存与调度,提升推理效率。
- 吞吐量提升至每小时3600请求,延迟降低至8.7秒以内。
- 适合金融风控、合规审查等高要求领域使用。
反欺诈与反洗钱(AML)是大语言模型(LLM)的重要应用领域,但其服务需求与通用对话任务差异显著。合规提示通常具有前缀密集、格式约束强、证据丰富等特点,包含可复用的政策指令、风险分类体系、交易或文档上下文,以及如JSON标签或风险因子等结构化输出。这些特性使得前缀复用、键值缓存效率、运行时调优、模型编排和输出验证成为关键系统挑战。本文提出一套面向欺诈与AML场景的工作负载感知型LLMOps部署栈,基于自托管开源模型如Meta Llama和阿里巴巴Qwen,集成vLLM式调优、分页注意力(PagedAttention)、自动前缀缓存、多适配器服务、适配器与提示长度感知批处理、休眠/唤醒生命周期管理、推测解码及可选的预填充/解码分离机制。为避免暴露机构敏感数据,采用公开合成数据集(如IBM AML和SAML-D),将其转化为含重用政策文本、交易证据、类型定义和格式约束输出的前缀密集型提示。同时引入基于LLM的质检机制,结合确定性合规检查、参考指标、专家校准数据及多评标评分体系。在公开合成AML任务与受控基准测试中,工作负载感知调优使吞吐量从612–650提升至3,600请求/小时,P99延迟从31–38秒降至6.4–8.7秒,GPU利用率由12%升至78%。结果表明,监管类LLM性能不仅是模型选择问题,更是工作负载设计、服务优化与质量管控的综合挑战。
原文摘要 · Abstract (English)
Fraud detection and anti-money-laundering (AML) compliance are high-value domains for large language models (LLMs), but their serving requirements differ sharply from generic chat workloads. Compliance prompts are often prefix-heavy, schema-constrained, and evidence-rich, combining reusable policy instructions, risk taxonomies, transaction or document context, and short structured outputs such as JSON labels or risk factors. These properties make prefix reuse, KV-cache efficiency, runtime tuning, model orchestration, and output validation first-order systems concerns. This paper introduces a workload-aware LLMOps stack for fraud and AML workloads using self-hosted open-weight models such as Meta Llama and Alibaba Qwen. The stack combines vLLM-style runtime tuning, PagedAttention, Automatic Prefix Caching, multi-adapter serving, adapter and prompt-length-aware batching, sleep/wake lifecycle management, speculative decoding, and optional prefill/decode disaggregation. To avoid exposing institution-specific data, the reproducibility track converts public synthetic AML datasets, including IBM AML and SAML-D, into prefix-heavy compliance prompts with reusable policy text, transaction evidence, typology definitions, and schema-constrained outputs. We also incorporate an LLM-as-judge quality gate using deterministic compliance checks, reference metrics, expert-adjudicated calibration data where available, and multi-judge rubric scoring. Across public-synthetic AML workloads and controlled serving benchmarks, workload-aware tuning improved throughput from 612-650 to 3,600 requests/hour, reduced P99 latency from 31-38 seconds to 6.4-8.7 seconds, and increased GPU utilization from 12% to 78%. These results show that regulated LLM performance is a workload-design, serving-optimization, and quality-gating problem, not only a model-selection problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。