arXiv:2602.20696cs.AI2026-02

无需训练,用提示词对比让大模型在测试时自动变可靠。

PromptCD: Test-Time Behavior Enhancement via Polarity-Prompt Contrastive Decoding

  • 用正负提示词对比模型输出概率,引导行为改进。
  • 在3H标准上提升大模型表现,效果显著且无需重训练。
  • 适用于图文模型和语言模型,适合追求高效对齐的开发者。

可靠的AI系统需要大语言模型(LLMs)的行为与人类偏好一致。然而,现有对齐方法多依赖训练时的高质量数据,成本高昂。尽管对比解码可利用模型内部分布提升特定能力,但适用范围有限。本文提出测试时行为增强方法Polarity-Prompt Contrastive Decoding(PromptCD),将对比解码推广至更广泛的增强场景。PromptCD为特定行为构建正负引导提示,通过对比模型响应——包括大模型的词级概率分布和视觉语言模型(VLMs)的视觉注意力模式——强化理想输出。该方法不需额外训练,适用于LLMs和VLMs。在“3H”对齐目标(助人、诚实、无害)上的实验表明,后训练模型可在测试时实现显著且持续的行为提升。对VLMs的分析显示,PromptCD通过强化与行为一致的视觉定位,显著提高视觉问答性能。结果表明,PromptCD是一种简单、通用、低成本的跨模态行为控制策略。

原文摘要 · Abstract (English)

Reliable AI systems require large language models (LLMs) to exhibit behaviors aligned with human preferences and values. However, most existing alignment approaches operate at training time and rely on additional high-quality data, incurring significant computational and annotation costs. While recent work has shown that contrastive decoding can leverage a model's internal distributions to improve specific capabilities, its applicability remains limited to narrow behavioral scopes and scenarios. In this work, we introduce Polarity-Prompt Contrastive Decoding (PromptCD), a test-time behavior control method that generalizes contrastive decoding to broader enhancement settings. PromptCD constructs paired positive and negative guiding prompts for a target behavior and contrasts model responses-specifically token-level probability distributions in LLMs and visual attention patterns in VLMs-to reinforce desirable outcomes. This formulation extends contrastive decoding to a wide range of enhancement objectives and is applicable to both LLMs and Vision-Language Models (VLMs) without additional training. For LLMs, experiments on the "3H" alignment objectives (helpfulness, honesty, and harmlessness) demonstrate consistent and substantial improvements, indicating that post-trained models can achieve meaningful self-enhancement purely at test time. For VLMs, we further analyze contrastive effects on visual attention, showing that PromptCD significantly improves VQA performance by reinforcing behavior-consistent visual grounding. Collectively, these results highlight PromptCD as a simple, general, and cost-efficient strategy for reliable behavior control across modalities.

行为对齐测试时优化对比解码大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。