通过调整生成时的标记序列，让大模型少拒答敏感问题。
Derailing Non-Answers via Logit Suppression at Output Subspace Boundaries in RLHF-Aligned Language Models
- 在推理时抑制特定换行符和结束标记，改变生成路径。
- 使敏感问题的回答率提升，且不影响常规任务表现。
- 无需训练或改模型，适合想降低拒答率的部署者。
我们提出一种方法，在不修改模型权重或提示词的情况下，降低大语言模型对敏感内容的拒答率。基于观察发现，某些模型在拒绝回答前常出现特定标记序列：链式思维起始标记后的结束序列标记。该方法无需数据集、参数更新或训练，仅通过生成阶段调整词元概率实现。在官方DeepSeek-R1蒸馏模型上的实验表明，这些干预显著提升了对敏感问题的实质性回答比例，同时未影响标准基准测试性能。研究结果表明，可通过在生成过程特定节点阻断拒答子空间，有效规避拒绝行为。
原文摘要 · Abstract (English)
We introduce a method to reduce refusal rates of large language models (LLMs) on sensitive content without modifying model weights or prompts. Motivated by the observation that refusals in certain models were often preceded by the specific token sequence of a token marking the beginning of the chain-of-thought (CoT) block (<think>) followed by a double newline token (\n\n), we investigate the impact of two simple formatting adjustments during generation: suppressing \n\n after <think> and suppressing the end-of-sequence token after the end of the CoT block (</think>). Our method requires no datasets, parameter changes, or training, relying solely on modifying token probabilities during generation. In our experiments with official DeepSeek-R1 distillations, these interventions increased the proportion of substantive answers to sensitive prompts without affecting performance on standard benchmarks. Our findings suggest that refusal behaviors can be circumvented by blocking refusal subspaces at specific points in the generation process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。