用大模型模拟用户搜索时的思维过程,让搜索行为更像真人。
Exploring Human-Like Thinking in Search Simulations with Large Language Models
- 让大模型先‘思考’再执行搜索动作,模仿人类认知流程。
- 引入真实用户思考数据,提升模拟行为的真实性。
- 适合研究搜索行为建模与智能系统评估的学者参考。
模拟用户搜索行为是信息检索中的关键任务,可用于用户行为建模、数据增强和系统评估。近年来,大语言模型(LLMs)为生成类人搜索行为(如查询、浏览、点击)提供了新可能。本文探索将类人思维融入搜索模拟,通过提示大模型在执行搜索动作前先模拟用户的隐性认知过程。由于现有搜索数据集缺乏用户思考内容,我们开展用户研究,收集了一个包含显式思考的新数据集。研究考察了引入类人思维对模拟性能的影响,并采用监督微调(SFT)训练大模型同时模仿人类思维与行为。实验从两个维度评估大模型在用户模拟中的应用:(1) 是否显式包含思考,(2) 是否在增强思考的数据集上进行微调。结果表明,融合类人思维在模拟中具备可行性与潜力,尽管部分指标提升有限。本工作为推进搜索模拟中的用户行为建模提供了新思路。
原文摘要 · Abstract (English)
Simulating user search behavior is a critical task in information retrieval, which can be employed for user behavior modeling, data augmentation, and system evaluation. Recent advancements in large language models (LLMs) have opened up new possibilities for generating human-like actions including querying, browsing, and clicking. In this work, we explore the integration of human-like thinking into search simulations by leveraging LLMs to simulate users' hidden cognitive processes. Specifically, given a search task and context, we prompt LLMs to first think like a human before executing the corresponding action. As existing search datasets do not include users' thought processes, we conducted a user study to collect a new dataset enriched with users' explicit thinking. We investigate the impact of incorporating such human-like thinking on simulation performance and apply supervised fine-tuning (SFT) to teach LLMs to emulate both human thinking and actions. Our experiments span two dimensions in leveraging LLMs for user simulation: (1) with or without explicit thinking, and (2) with or without fine-tuning on the thinking-augmented dataset. The results demonstrate the feasibility and potential of incorporating human-like thinking in user simulations, though performance improvements on some metrics remain modest. We believe this exploration provides new avenues and inspirations for advancing user behavior modeling in search simulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。