让大模型生成更安全,通过语义嵌入动态过滤不当内容
DIESEL -- Dynamic Inference-Guidance via Evasion of Semantic Embeddings in LLMs
- 基于隐空间相似度重排候选词,实现轻量级推理引导
- 在对抗性攻击下仍能有效防止越狱输出,提升响应安全性
- 可通用过滤不想要的内容,适合多种安全与合规场景
近年来,大语言模型在闲聊等任务中取得显著进展,推动虚拟助手等领域发展。然而,其生成内容常偏离人类价值观(如伦理标准、安全规范),导致潜在不安全或不恰当输出。现有解决方法多需昂贵训练或大幅增加推理时间。本文提出DIESEL,一种轻量级推理引导技术,可无缝集成至任意自回归大模型,通过在隐空间中计算候选词与预定义负面概念的相似度,动态过滤不当语义内容。该方法可独立运行或作为额外防护层,实现响应安全增强。实验表明,DIESEL在主流对话模型上表现优异,即使在挑战性越狱攻击下也能有效防御。同时,其泛化能力显示,该技术也可用于非安全场景的通用内容过滤。
原文摘要 · Abstract (English)
In recent years, large language models (LLMs) have had great success in tasks such as casual conversation, contributing to significant advancements in domains like virtual assistance. However, they often generate responses that are not aligned with human values (e.g., ethical standards, safety), leading to potentially unsafe or inappropriate outputs. While several techniques have been proposed to address this problem, they come with a cost, requiring computationally expensive training or dramatically increasing the inference time. In this paper, we present DIESEL, a lightweight inference-guidance technique that can be seamlessly integrated into any autoregressive LLM to semantically filter undesired concepts from the response. DIESEL can function either as a standalone safeguard or as an additional layer of defense, enhancing response safety by reranking the LLM's proposed tokens based on their similarity to predefined negative concepts in the latent space. Our evaluation demonstrates DIESEL's effectiveness on state-of-the-art conversational models, even in adversarial jailbreaking scenarios that challenge response safety. We also highlight DIESEL's generalization capabilities, showing that it can be used in use cases other than safety, providing general-purpose response filtering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。