提出P3方法，让大模型一次生成多个位置的词概率，提升零样本分类鲁棒性。
Beyond the Next Token: Towards Prompt-Robust Zero-Shot Classification via Efficient Multi-Token Prediction
- 一次预测多个位置的词概率，模拟完整生成路径
- 准确率提升，提示词间性能标准差降低98%
- 无需提示词也能保持良好表现，减少人工调参
零样本文本分类通常依赖提示工程，但大语言模型对提示敏感，微小改动会导致性能剧烈波动。我们发现这主要源于现有方法仅关注下一个词的概率。为此提出并行占位预测（P3），在单次推理中预测多个位置的词概率，模拟完整生成路径。实验显示，该方法提升分类准确率，提示词间性能标准差下降最高达98%，显著增强鲁棒性。即使不使用提示词，P3仍能保持相近性能，大幅减少对提示工程的依赖。
原文摘要 · Abstract (English)
Zero-shot text classification typically relies on prompt engineering, but the inherent prompt brittleness of large language models undermines its reliability. Minor changes in prompt can cause significant discrepancies in model performance. We attribute this prompt brittleness largely to the narrow focus on nexttoken probabilities in existing methods. To address this, we propose Placeholding Parallel Prediction (P3), a novel approach that predicts token probabilities across multiple positions and simulates comprehensive sampling of generation paths in a single run of a language model. Experiments show improved accuracy and up to 98% reduction in the standard deviation across prompts, boosting robustness. Even without a prompt, P3 maintains comparable performance, reducing the need for prompt engineering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。