用推理训练提升安全检测效果,推理时却无需生成过程。
DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

- 训练时用推理监督,推理时只输出结构化标签。
- 在双侧安全测试中平均F1达0.878,4B模型超8B基线。
- 适合需要低延迟、高安全性的实际部署场景。
部署于开放世界应用的大语言模型需具备既能抵御复杂风险又足够高效的安全部署机制。现有安全部署方案在轻量级分类模型(高效但难处理隐蔽意图、模糊语义与边界判断)与基于推理的防护机制(判断更准但引入额外生成与延迟)之间存在权衡。本文提出DT-Guard,一种基于「推理主动训练、推理无推理」范式的安全防护模型。核心思路是在训练阶段引入推理监督,而推理阶段仅输出结构化安全标签。该模型将安全判断建模为渐进式决策流程:意图→类别→安全,构建包含意图标签、风险类别、安全标签及结构化推理轨迹的意图驱动数据集。为进一步增强困难样本鲁棒性,提出滚动引导渐进式困难样本优化(RG-PHO),利用多轮次一致性识别稳定掌握、持续失败与偏好不稳定的样本,并分别施以监督与偏好优化。推理时直接生成结构化标签,无需显式推理痕迹,保持部署效率。在提示侧与响应侧安全基准测试中,DT-Guard分别达到0.886和0.870的平均F1。仅使用4B主干模型即实现双侧平均F1 0.878,超越强基线8B模型。结果表明,推理监督可有效内化为低延迟安全判别能力。
原文摘要 · Abstract (English)
Large language models deployed in open-world applications require safety guardrails that are both robust to complex risks and efficient enough for low-latency runtime moderation. Existing guardrails face a practical trade-off between lightweight classification-based models, which are efficient but often struggle with concealed intent, ambiguous semantics, and borderline safety decisions, and reasoning-based guards, which improve judgment quality but introduce additional token generation and inference latency. We present DT-Guard, a content safety guardrail model based on a Reasoning-Active Training, Reasoning-Free Inference paradigm. The key idea is to use reasoning supervision during training while emitting only structured safety labels at inference time. DT-Guard formulates safety judgment as a progressive decision process, Intent - Category - Safety, and constructs an intent-driven dataset with intent labels, risk categories, safety labels, and structured reasoning trajectories. To further improve hard-case robustness, we propose Rollout-Guided Progressive Hard-Case Optimization (RG-PHO), which uses multi-rollout consistency to identify stably mastered, persistently failed, and preference-unstable samples, and applies targeted supervised and preference optimization accordingly. At inference time, DT-Guard directly generates structured labels without explicit reasoning traces, preserving deployment efficiency. Experiments on prompt-side and response-side safety benchmarks show that DT-Guard achieves average F1 scores of 0.886 and 0.870, respectively. With only a 4B backbone, it reaches a dual-side average F1 of 0.878, outperforming strong 8B guardrail baselines. These results demonstrate that reasoning supervision can be effectively internalized into low-latency safety discrimination.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。