用智能搜索优化大模型输出，无需重训练就能提升回答质量。
MEMETRON: Metaheuristic Mechanisms for Test-time Response Optimization of Large Language Models
- 将生成过程看作寻优问题，用遗传与神经算法找最佳答案。
- 在人类偏好任务上超越传统方法，响应质量显著提升。
- 适合需要高质量输出但无法调参的场景，如客服、写作辅助。
大型语言模型（LLMs）广泛应用于开放式和结构化任务，但其推理行为仍由贪婪搜索、采样或重排序等启发式策略主导，控制力弱且不针对具体目标优化。我们提出MEMETRON，一种任务无关的框架，将LLM解码建模为离散黑箱优化问题。该框架利用混合元启发式算法GENETRON和ANNETRON，在奖励模型引导下搜索响应空间，并通过LLM自身执行上下文操作。该方法无需模型重训练或梯度信息，即可高效发现高奖励响应。框架模块化，可泛化至多种任务，仅需奖励函数和轻量提示模板。我们在关键的人类偏好对齐任务上评估，结果表明其显著优于标准解码与重排序方法，展示了其在不重训练的前提下提升对齐性的潜力。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used for both open-ended and structured tasks, yet their inference-time behavior is still largely dictated by heuristic decoding strategies such as greedy search, sampling, or reranking. These methods provide limited control and do not explicitly optimize for task-specific objectives. We introduce MEMETRON, a task-agnostic framework that formulates LLM decoding as a discrete black-box optimization problem. MEMETRON leverages hybrid metaheuristic algorithms, GENETRON and ANNETRON, to search the response space, guided by reward models and contextual operations performed by the LLM itself. This approach enables efficient discovery of high-reward responses without requiring model retraining or gradient access. The framework is modular and generalizes across diverse tasks, requiring only a reward function and lightweight prompt templates. We evaluate our framework on the critical human preference alignment task and demonstrate that it significantly outperforms standard decoding and reranking methods, highlighting its potential to improve alignment without model retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。