用稀疏自编码器让大模型安全分类更高效，只需两个约束平面即可达到高精度。
Geometry-Guided Constraint Learning for LLM Safety Classification
- 利用稀疏自编码器提取特征，使安全约束数量从需调参变为固定为2
- 在Qwen3.5-9B上12/14类别达96-99%准确率，显著减少调参成本
- 基于几何结构设计可学习锥形约束，适合追求高效安全分类的研究者
安全作为多面体（SaP）在大语言模型隐藏空间中学习线性半空间约束，但需对每类手动调整约束数量K。本文发现，通过稀疏自编码器（SAE）特征提取可解决此问题：在Qwen3.5-9B上，将K设为2即成为最优配置，覆盖14个类别中的12个，于BeaverTails分类基准上实现每个类别96-99%的准确率，大幅减少对大量参数组合（K=4-25且随机初始化）的试错需求。该结果与线性表示假设一致，暗示在此设定下，安全边界在SAE特征空间中具有低维线性结构。基于此几何视角，我们提出一种可学习开口角度的锥形约束，其训练过程通过三阶段稳定优化，有效适配各类别簇集中度。
原文摘要 · Abstract (English)
Safety as Polytope (SaP) learns linear half-space constraints in LLM hidden space but requires per-category tuning of the constraint count K. We show that sparse autoencoder (SAE) feature extraction resolves this: K=2 becomes optimal for 12/14 categories on Qwen3.5-9B, achieving 96-99% accuracy per category on our BeaverTails classification benchmark, largely eliminating the need for exhaustive sweeps (K=4-25 with random initialization). This convergence to two planes is consistent with the Linear Representation Hypothesis, providing suggestive evidence that safety boundaries in this setting admit a low-dimensional linear description in the SAE feature space. Building on this geometric perspective, we introduce a cone constraint whose learnable aperture adapts to each category's cluster concentration, stabilized by a three-phase training
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。