arXiv:2412.14454cs.IRcs.CL2024-12被引 14

研究发现:选对提示词比用更长的提示词更重要。

Are Longer Prompts Always Better? Prompt Selection in Large Language Models for Recommendation Systems

  • 根据数据特征选提示词,而非盲目加长
  • 90个提示词实验表明无万能最优解
  • 新方法用少量验证数据高效选优提示

基于大语言模型的推荐系统(LLM-RS)可通过自然语言提示,在数据稀缺场景下有效预测用户偏好,尤其适用于冷启动与跨域问题。然而,提示词的选择直接影响推荐准确率,现有研究缺乏明确筛选标准。本文分析了90种来自先前工作的提示词,通过450次实验在五个真实数据集上探究提示词与数据特征的关系。结果表明,不存在始终表现最优的提示词,提示选择必须依赖数据特性。为此,提出一种基于数据特征的提示选择方法,仅需少量验证数据即可实现更高准确率。同时引入高性能且低成本的大语言模型,显著降低探索成本,兼顾效率与精度。本研究为高效、精准的LLM-RS提供实用指导。

原文摘要 · Abstract (English)

In large language models (LLM)-based recommendation systems (LLM-RSs), accurately predicting user preferences by leveraging the general knowledge of LLMs is possible without requiring extensive training data. By converting recommendation tasks into natural language inputs called prompts, LLM-RSs can efficiently solve issues that have been difficult to address due to data scarcity but are crucial in applications such as cold-start and cross-domain problems. However, when applying this in practice, selecting the prompt that matches tasks and data is essential. Although numerous prompts have been proposed in LLM-RSs and representing the target user in prompts significantly impacts recommendation accuracy, there are still no clear guidelines for selecting specific prompts. In this paper, we categorize and analyze prompts from previous research to establish practical prompt selection guidelines. Through 450 experiments with 90 prompts and five real-world datasets, we examined the relationship between prompts and dataset characteristics in recommendation accuracy. We found that no single prompt consistently outperforms others; thus, selecting prompts on the basis of dataset characteristics is crucial. Here, we propose a prompt selection method that achieves higher accuracy with minimal validation data. Because increasing the number of prompts to explore raises costs, we also introduce a cost-efficient strategy using high-performance and cost-efficient LLMs, significantly reducing exploration costs while maintaining high prediction accuracy. Our work offers valuable insights into the prompt selection, advancing accurate and efficient LLM-RSs.

提示词选择推荐系统大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。