让大模型安全防护更高效可查，用连续隐状态代替文字推理
LatentGuard: Efficient and Inspectable Latent Reasoning for LLM Safeguards
- 将安全判断的推理过程压缩为紧凑的连续隐状态，降低计算开销
- 相比原有方法，推理令牌数从268.56降至1.60，准确率提升至84.91
- 可按需生成审计文本，保障部署时的可解释性与安全性
基于推理的守护模型能提升大模型的安全性，但每次交互都生成显式推理过程会带来高昂的部署成本。尽管隐状态推理方法通过将推理过程嵌入连续表示来减少令牌生成，但在安全防护领域仍研究不足，且缺乏可检查的部署接口。本文提出LatentGuard，一种高效且可检查的守护框架，首次将连续隐状态推理引入守护模型。该框架采用分阶段课程训练，逐步将任务对齐的文本推理压缩为紧凑的隐状态，使安全判定可直接由连续表征完成。为保持可解释性，设计独立的辅助解码器，在需要时生成紧凑审计文本，避免影响标准推理路径。实验表明，LatentGuard-8B将平均加权F1从83.95提升至84.91，关键路径推理令牌数由268.56降至1.60。其审计解码器获得85.75的审计效用得分，验证了该路径在可部署性与可解释性上的可行性。
原文摘要 · Abstract (English)
Reasoning-based guard models improve LLM safeguards, but decoding explicit rationales for every interaction makes them costly to deploy. Although latent-reasoning methods reduce token generation by moving reasoning into continuous states, they remain underexplored for safety moderation and lack an inspection interface for deployment. In this paper, we propose LatentGuard, an efficient and inspectable safeguard framework that brings continuous latent reasoning to guard models. LatentGuard uses a staged curriculum to progressively compress task-aligned textual rationales into compact latent states, enabling safety verdicts to be predicted directly from continuous representations. To preserve inspectability, an isolated auxiliary decoder generates compact audit artifacts on demand, keeping rationale generation off the standard inference path. Experiments show that LatentGuard-8B improves mean weighted F1 from 83.95 to 84.91 over GuardReasoner-8B, while reducing critical-path reasoning cost from 268.56 generated rationale tokens to 1.60 latent reasoning tokens. Its audit decoder achieves an audit utility score of 85.75, demonstrating an efficient and inspectable path toward deployable LLM safeguards.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。