不训练模型，仅在生成时优化输入嵌入来消除毒性内容。
Test-Time Detoxification without Training or Learning Anything
- 通过零阶优化直接调整输入嵌入，引导生成更安全文本。
- 在多个模型和提示下实现显著毒性强降低，且保持生成质量。
- 无需训练或中间计算，适用于黑盒模型，适合实际部署。
大语言模型在处理良性输入时仍可能生成有毒或不当文本，部署时存在安全风险。为提升安全性与用户信任，需在不牺牲生成质量的前提下减少有害内容。现有方法多依赖重训练、梯度或学习的辅助组件，成本高且难以跨模型迁移或应用于完全黑盒场景。本文提出一种测试时的去毒方法：通过零阶优化近似生成毒性对输入嵌入的梯度，仅用少量下降步数将生成结果导向更少毒性的延续。该方法仅需输入嵌入、毒性评分函数及模型前向推理，无需训练或中间计算访问。实验表明，该方法在多种模型和提示下均实现稳健的毒性降低，在多数场景中取得最优的毒性-质量权衡。本工作将词嵌入视为有效控制变量，推动黑盒优化在自回归语言模型中的应用，实现无需训练的可扩展、安全文本生成。
原文摘要 · Abstract (English)
Large language models can produce toxic or inappropriate text even for benign inputs, creating risks when deployed at scale. Detoxification is therefore important for safety and user trust, particularly when we want to reduce harmful content without sacrificing the model's generation quality. Many existing approaches rely on model retraining, gradients, or learned auxiliary components, which can be costly and may not transfer across model families or to truly black-box settings. We introduce a test-time procedure that approximates the gradient of completion toxicity with respect to the input embeddings and uses a small number of descent steps to steer generation toward less toxic continuations. This is achieved with zeroth-order optimization that requires only access to input embeddings, a toxicity scoring function, and forward evaluations of the model. Empirically, the approach delivers robust toxicity reductions across models and prompts and, in most settings, achieves the best overall toxicity-quality trade-off. More broadly, our work positions word embeddings as effective control variables and encourages wider use of black-box optimization to guide autoregressive language models toward scalable, safer text generation, without requiring any training or access to intermediate computations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。