通过动态聚类与动量机制，提升大模型提示词优化的稳定性和精度。
C-MOP: Integrating Momentum and Boundary-Aware Clustering for Enhanced Prompt Evolution
- 基于边界感知采样和动量聚类，抑制噪声更新信号。
- 相比现有方法，平均提升1.58%至3.35%，3B模型超越70B专用模型。
- 适合需要高效微调提示词的NLP研究者与工程应用者。
自动提示词优化是提升大语言模型性能的有前景方向，但现有方法常受噪声和冲突更新信号干扰。本文提出C-MOP（基于聚类的动量优化提示），通过边界感知对比采样（BACS）和动量引导语义聚类（MGSC）实现优化稳定。BACS利用批处理信息挖掘三元组特征——难负样本、锚点与边界对，精准刻画正负样本的典型表征与决策边界；MGSC引入带时间衰减的文本动量机制，从迭代中的波动梯度中提炼持久共识。大量实验表明，C-MOP持续优于当前最优基线如PromptWizard和ProTeGi，平均提升1.58%与3.35%。尤为突出的是，一个仅激活30亿参数的通用模型性能超越700亿参数的领域专用密集模型，彰显其在精确提示词演化中的有效性。代码已公开于https://github.com/huawei-noah/noah-research/tree/master/C-MOP。
原文摘要 · Abstract (English)
Automatic prompt optimization is a promising direction to boost the performance of Large Language Models (LLMs). However, existing methods often suffer from noisy and conflicting update signals. In this research, we propose C-MOP (Cluster-based Momentum Optimized Prompting), a framework that stabilizes optimization via Boundary-Aware Contrastive Sampling (BACS) and Momentum-Guided Semantic Clustering (MGSC). Specifically, BACS utilizes batch-level information to mine tripartite features--Hard Negatives, Anchors, and Boundary Pairs--to precisely characterize the typical representation and decision boundaries of positive and negative prompt samples. To resolve semantic conflicts, MGSC introduces a textual momentum mechanism with temporal decay that distills persistent consensus from fluctuating gradients across iterations. Extensive experiments demonstrate that C-MOP consistently outperforms SOTA baselines like PromptWizard and ProTeGi, yielding average gains of 1.58% and 3.35%. Notably, C-MOP enables a general LLM with 3B activated parameters to surpass a 70B domain-specific dense LLM, highlighting its effectiveness in driving precise prompt evolution. The code is available at https://github.com/huawei-noah/noah-research/tree/master/C-MOP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。