通过分析模型表示空间，提升角色扮演模型拒绝不当请求的能力。
Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing
- 分析模型在不同冲突请求下的表示空间，发现拒绝与直接回应区域。
- 提出轻量级编辑方法，将冲突请求映射到拒绝区域，拒绝准确率显著提升。
- 适合关注安全可控对话系统的研究人员与工程师。
角色扮演代理（RPAs）在各类应用中表现优异，但在面对与角色知识冲突的复杂请求时，常无法正确识别并恰当拒绝。为此，我们构建了一个评估基准，涵盖上下文知识冲突、参数知识冲突及非冲突请求，用于评估 RPAs 识别冲突并合理拒绝的能力，同时避免过度拒绝。大量实验表明，多数 RPAs 在不同冲突类型上表现差异显著。进一步的表示层分析揭示，模型前向表示中存在拒绝区与直接回应区，影响最终响应行为。为此，我们提出一种轻量级表示编辑方法，可将冲突请求有效引导至拒绝区，从而增强拒绝准确性。实验验证该方法在提升拒绝能力的同时，保持了模型原有的角色扮演性能。
原文摘要 · Abstract (English)
Role-Playing Agents (RPAs) have shown remarkable performance in various applications, yet they often struggle to recognize and appropriately respond to hard queries that conflict with their role-play knowledge. To investigate RPAs' performance when faced with different types of conflicting requests, we develop an evaluation benchmark that includes contextual knowledge conflicting requests, parametric knowledge conflicting requests, and non-conflicting requests to assess RPAs' ability to identify conflicts and refuse to answer appropriately without over-refusing. Through extensive evaluation, we find that most RPAs behave significant performance gaps toward different conflict requests. To elucidate the reasons, we conduct an in-depth representation-level analysis of RPAs under various conflict scenarios. Our findings reveal the existence of rejection regions and direct response regions within the model's forwarding representation, and thus influence the RPA's final response behavior. Therefore, we introduce a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model's refusal accuracy. The experimental results validate the effectiveness of our editing method, improving RPAs' refusal ability of conflicting requests while maintaining their general role-playing capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。