用检索增强生成提升大模型出行方式预测准确率
Benchmarking Retrieval-Augmented Generation Strategies for Large Language Model-Based Travel Mode Choice Prediction
- 将检索增强生成融入大模型,结合实证数据做出行选择预测
- 最优组合下准确率达80.8%,显著优于传统方法
- 适合交通规划与智能决策领域研究者参考
准确预测出行方式对有效交通规划至关重要,但传统统计和机器学习模型受限于僵化假设、上下文推理能力弱及可迁移性差。本研究探索大语言模型(LLMs)作为更灵活、具上下文感知能力的出行方式预测新范式,并通过检索增强生成(RAG)将其预测结果锚定在实证数据上。我们构建了一个模块化框架,集成四种检索策略:基础RAG、平衡检索RAG、基于交叉编码器重排序的RAG,以及结合平衡检索与交叉编码器重排序的RAG。在三种LLM架构(OpenAI GPT-4o、o4-mini 和 o3)上评估其性能,分析模型推理能力与检索方法的交互作用。基于2023年普吉特海湾地区家庭出行调查数据开展实验。结果显示,RAG显著提升各类模型的预测准确率;其中,GPT-4o搭配平衡检索与交叉编码器重排序时达到最高准确率80.8%,优于传统统计与机器学习基线。此外,基于LLM的模型展现出更强的零样本迁移能力。研究揭示了大模型推理能力与检索策略间的关键互动,强调需根据模型特性匹配检索策略,以最大化大模型在出行行为建模中的潜力。
原文摘要 · Abstract (English)
Accurately predicting travel mode choice is essential for effective transportation planning, yet traditional statistical and machine learning models are constrained by rigid assumptions, limited contextual reasoning, and reduced transferability. This study explores the potential of Large Language Models (LLMs) as a more flexible and context-aware approach to travel mode choice prediction, enhanced by Retrieval-Augmented Generation (RAG) to ground predictions in empirical data. We develop a modular framework for integrating RAG into LLM-based travel mode choice prediction and evaluate four retrieval strategies: basic RAG, RAG with balanced retrieval, RAG with a cross-encoder for re-ranking, and RAG with balanced retrieval and a cross-encoder for re-ranking. These strategies are tested across three LLM architectures (OpenAI GPT-4o, o4-mini, and o3) to examine the interaction between model reasoning capabilities and retrieval methods. Using the 2023 Puget Sound Regional Household Travel Survey data, we conduct a series of experiments to evaluate model performance. The results demonstrate that RAG substantially enhances predictive accuracy across a range of models. Notably, the GPT-4o model combined with balanced retrieval and cross-encoder re-ranking achieves the highest accuracy of 80.8%, exceeding that of conventional statistical and machine learning baselines. Furthermore, LLM-based models exhibit superior zero-shot transfer abilities relative to these baselines. Findings highlight the critical interplay between LLM reasoning capabilities and retrieval strategies, demonstrating the importance of aligning retrieval strategies with model capabilities to maximize the potential of LLM-based travel behavior modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。