用模型内部表示排序回复，提升大模型生成内容的安全性
Advancing LLM Safe Alignment with Safety Representation Ranking
- 基于模型中间层表示进行安全排序，不依赖文本直接判断
- 在多个评测中对恶意提示的鲁棒性显著提升
- 适合关注大模型安全对齐的研究者和开发者
大语言模型在多项任务中取得突破性进展，但其生成有害内容的风险引发广泛关注。现有安全评估方法多直接分析文本输出，忽视了模型内部表示中蕴含的丰富信息。本文提出安全表示排序（SRR），一种基于列表的排序框架，利用LLM自身的隐藏状态筛选安全回复。SRR通过中间Transformer表示编码指令与候选回复，并采用轻量级相似度评分器进行排序。该方法直接利用模型内部状态与列表级别监督，捕捉细微的安全信号。在多个基准测试中，SRR显著提升了对对抗性提示的鲁棒性。代码将于发表后公开。
原文摘要 · Abstract (English)
The rapid advancement of large language models (LLMs) has demonstrated milestone success in a variety of tasks, yet their potential for generating harmful content has raised significant safety concerns. Existing safety evaluation approaches typically operate directly on textual responses, overlooking the rich information embedded in the model's internal representations. In this paper, we propose Safety Representation Ranking (SRR), a listwise ranking framework that selects safe responses using hidden states from the LLM itself. SRR encodes both instructions and candidate completions using intermediate transformer representations and ranks candidates via a lightweight similarity-based scorer. Our approach directly leverages internal model states and supervision at the list level to capture subtle safety signals. Experiments across multiple benchmarks show that SRR significantly improves robustness to adversarial prompts. Our code will be available upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。