用激活探针高效检测高风险对话，节省算力六倍
Detecting High-Stakes Interactions with Activation Probes
- 用合成数据训练激活探针，捕捉潜在高危交互
- 在真实数据上表现接近大模型监控，但提速百万倍
- 适合资源受限场景，可作低成本初步筛查
监控大型语言模型的安全部署至关重要。本文聚焦于通过激活探针检测‘高风险’交互——即文本暗示可能造成重大危害的对话。我们评估了多种基于合成数据训练的探针架构，在多样化的分布外真实数据上表现出稳健泛化能力。探针性能与提示或微调的中型大模型监控相当，但计算开销降低六数量级，得益于复用被监控模型的激活值。实验还表明，可构建资源感知的分层监控系统：探针作为高效初筛，标记需进一步分析的案例。代码与新合成数据集已开源至 https://github.com/arrrlex/models-under-pressure。
原文摘要 · Abstract (English)
Monitoring is an important aspect of safely deploying Large Language Models (LLMs). This paper examines activation probes for detecting ``high-stakes'' interactions -- where the text indicates that the interaction might lead to significant harm -- as a critical, yet underexplored, target for such monitoring. We evaluate several probe architectures trained on synthetic data, and find them to exhibit robust generalization to diverse, out-of-distribution, real-world data. Probes' performance is comparable to that of prompted or finetuned medium-sized LLM monitors, while offering computational savings of six orders-of-magnitude. These savings are enabled by reusing activations of the model that is being monitored. Our experiments also highlight the potential of building resource-aware hierarchical monitoring systems, where probes serve as an efficient initial filter and flag cases for more expensive downstream analysis. We release our novel synthetic dataset and the codebase at https://github.com/arrrlex/models-under-pressure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。