用图结构提升长文档合规检测效率,小模型也能达大模型效果。
Long SKILL Compliance as Logical Reasoning: Closure-Grounded Detection with Scaling-Guided On-Policy Distillation

- 构建双层依赖图表示业务规则,支持精准推理与溯源。
- 相比基线最高提升12.8%准确率,推理耗token减少64.3%。
- 发现模型性能随规则复杂度有规律可循,适合资源受限场景优化。
企业业务场景日益复杂,长篇SKILL文档在智能代理系统中广泛应用,带来合规检测新挑战:大模型推理成本高,小模型则难以保持准确率。为此,我们提出SkillCDG,一种基于图的长SKILL合规检测框架。该框架将复杂业务策略建模为两层约束依赖图,上层用于场景路由的SKILL描述索引,下层刻画每个SKILL内原子约束间的依赖关系。推理时通过两级检索结合依赖闭包实现合规判断与源头追溯。在三个企业数据集及两个可控公开基准上的实验表明,SkillCDG在检测F1上较基线最高提升12.8个百分点,同时最大降低64.3%的token消耗。进一步分析发现,策略图复杂度、模型规模与检测性能间存在内在关联。对同一模型家族四个检查点的对比实验验证了端到端检测正确率呈现复杂度差异化的缩放规律,且依赖图中的复杂度度量能有效量化实例难度与模型性能提升潜力。基于此规律,我们采用自适应样本选择与在策略蒸馏,显著提升小模型的合规检测能力。
原文摘要 · Abstract (English)
The increasing complexity of enterprise business scenarios has promoted the widespread adoption of long SKILL documents in agent systems, posing new challenges for compliance detection: large models incur substantial inference costs, while small models may fail to maintain detection accuracy. To address this gap, we propose SkillCDG, a graph-based framework for long SKILL compliance detection. SkillCDG represents complex business policies as a two-layer constraint dependency graph, where the upper layer indexes SKILL descriptions for scenario routing and the lower layer captures dependencies among atomic constraints within each SKILL. During inference, two-level retrieval followed by dependency closure supports compliance judgment and source traceability. We comprehensively evaluate the framework on three enterprise datasets and two controlled public benchmark variants. Experimental results demonstrate that SkillCDG outperforms baseline methods by up to 12.8 percentage points in detection F1 score, while reducing token consumption by a maximum 64.3\%. Moreover, we further investigate the inherent relationships among policy-graph complexity, model scale, and detection performance. Comparative experiments conducted on four checkpoints from a single model family validate a concise and effective scaling trend: end-to-end detection correctness exhibits a complexity-differentiated scaling pattern, and the complexity metric derived from the constraint dependency graph can effectively quantify instance difficulty and the performance improvement potential of models. Leveraging this insightful scaling trend, we conduct adaptive training sample selection and adopt on-policy distillation to efficiently enhance the compliance detection capability of small-scale models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。