arXiv:2507.13525cs.IR2025-07中稿 · ACM RecSys2025 rep…被引 12

优化大模型推荐的提示词设计,提升个性化推荐效果与效率。

Revisiting Prompt Engineering: A Comprehensive Evaluation for LLM-based Personalized Recommendation

  • 对比23种提示词类型,发现重述指令、融入背景知识等三类最有效。
  • 低成本模型用复杂提示反而降低准确率,简单提示更优且省算力。
  • 适合隐私敏感或数据少的场景,指导如何平衡精度与成本。

大语言模型可通过自然语言提示执行推荐任务,相比传统方法在冷启动、跨域和零样本场景更具优势,并支持灵活输入与行为解释。本文聚焦单用户设置(不使用其他用户信息),适用于隐私敏感或数据受限场景。在此设定下,提示工程对控制模型输出至关重要。我们对8个公开数据集和12个LLM进行了大规模对比,涵盖23种提示类型,采用统计检验与线性混合效应模型评估准确率与推理开销。结果表明:对成本敏感的模型,重述指令、融合背景知识、简化推理过程三类提示尤为有效;对高性能模型,简单提示常优于复杂形式,且降低开销。而常见NLP提示风格如分步推理或使用推理模型,反而导致准确率下降。基于此,我们提出根据精度与成本需求选择提示与模型的实用建议。

原文摘要 · Abstract (English)

Large language models (LLMs) can perform recommendation tasks by taking prompts written in natural language as input. Compared to traditional methods such as collaborative filtering, LLM-based recommendation offers advantages in handling cold-start, cross-domain, and zero-shot scenarios, as well as supporting flexible input formats and generating explanations of user behavior. In this paper, we focus on a single-user setting, where no information from other users is used. This setting is practical for privacy-sensitive or data-limited applications. In such cases, prompt engineering becomes especially important for controlling the output generated by the LLM. We conduct a large-scale comparison of 23 prompt types across 8 public datasets and 12 LLMs. We use statistical tests and linear mixed-effects models to evaluate both accuracy and inference cost. Our results show that for cost-efficient LLMs, three types of prompts are especially effective: those that rephrase instructions, consider background knowledge, and make the reasoning process easier to follow. For high-performance LLMs, simple prompts often outperform more complex ones while reducing cost. In contrast, commonly used prompting styles in natural language processing, such as step-by-step reasoning, or the use of reasoning models often lead to lower accuracy. Based on these findings, we provide practical suggestions for selecting prompts and LLMs depending on the required balance between accuracy and cost.

提示工程个性化推荐大模型应用成本优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。