构建驾驶场景向量化检索与生成框架，提升智能驾驶决策效率。
Driving-RAG: Driving Scenarios Embedding, Search, and RAG Applications
- 将驾驶场景信息映射到向量空间，实现语义对齐的高效检索。
- 结合图结构重组织，显著提升大模型生成与场景的匹配度。
- 适用于复杂交互场景下的轨迹规划，适合自动驾驶研发团队。
驾驶场景数据在智能汽车和自动驾驶系统发展中日益关键。准确高效的场景数据检索对在线决策规划和离线场景生成仿真至关重要，有助于利用历史经验提升系统性能。随着大语言模型（LLMs）与检索增强生成（RAG）系统在自动驾驶中的应用，相关需求愈发迫切。本文提出Driving-RAG框架，解决场景数据嵌入、搜索及RAG应用中的挑战。嵌入模型在向量空间中对齐基础场景信息与场景距离度量；结合典型场景采样与层次可导航小世界（HNSW）算法，实现高效向量搜索，兼顾精度与速度；通过图知识重构机制增强生成内容与提示场景的相关性。在复杂交互场景（如匝道、交叉口）的典型轨迹规划任务中验证了框架有效性，展现出在RAG应用中的优势。
原文摘要 · Abstract (English)
Driving scenario data play an increasingly vital role in the development of intelligent vehicles and autonomous driving. Accurate and efficient scenario data search is critical for both online vehicle decision-making and planning, and offline scenario generation and simulations, as it allows for leveraging the scenario experiences to improve the overall performance. Especially with the application of large language models (LLMs) and Retrieval-Augmented-Generation (RAG) systems in autonomous driving, urgent requirements are put forward. In this paper, we introduce the Driving-RAG framework to address the challenges of efficient scenario data embedding, search, and applications for RAG systems. Our embedding model aligns fundamental scenario information and scenario distance metrics in the vector space. The typical scenario sampling method combined with hierarchical navigable small world can perform efficient scenario vector search to achieve high efficiency without sacrificing accuracy. In addition, the reorganization mechanism by graph knowledge enhances the relevance to the prompt scenarios and augment LLM generation. We demonstrate the effectiveness of the proposed framework on typical trajectory planning task for complex interactive scenarios such as ramps and intersections, showcasing its advantages for RAG applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。