LLM与数据管理双向融合,重塑大模型训练与数据处理范式。
A Survey of LLM $\times$ DATA
- 构建了LLM与数据管理的双向技术体系框架。
- 涵盖数据处理、存储、服务及智能数据操作全流程。
- 适合大模型研发与数据库系统优化方向的研究者参考。
大型语言模型(LLM)与数据管理(DATA)的融合正在快速重塑两个领域。本文全面综述二者间的双向关系:一方面,DATA4LLM涵盖大规模数据处理、存储与服务,为预训练、后训练、检索增强生成(RAG)及代理工作流等阶段提供高质量、多样化、及时的数据支持;包括可扩展的数据获取、去重、过滤、选择、领域混合与合成增强;数据存储聚焦高效格式、分布式异构存储层次、KV缓存管理与容错检查点;数据服务则应对RAG中的知识后处理、推理中的提示压缩与数据溯源、训练策略中的数据打包与打乱等挑战。另一方面,在LLM4DATA中,LLM正成为通用数据管理引擎,推动自动数据清洗、集成与发现,结构化/半结构化/非结构化数据推理,以及系统优化如配置调优、查询重写与异常诊断,依托检索增强提示、任务特化微调与多智能体协作等技术。
原文摘要 · Abstract (English)
The integration of large language model (LLM) and data management (DATA) is rapidly redefining both domains. In this survey, we comprehensively review the bidirectional relationships. On the one hand, DATA4LLM, spanning large-scale data processing, storage, and serving, feeds LLMs with high quality, diversity, and timeliness of data required for stages like pre-training, post-training, retrieval-augmented generation, and agentic workflows: (i) Data processing for LLMs includes scalable acquisition, deduplication, filtering, selection, domain mixing, and synthetic augmentation; (ii) Data Storage for LLMs focuses on efficient data and model formats, distributed and heterogeneous storage hierarchies, KV-cache management, and fault-tolerant checkpointing; (iii) Data serving for LLMs tackles challenges in RAG (e.g., knowledge post-processing), LLM inference (e.g., prompt compression, data provenance), and training strategies (e.g., data packing and shuffling). On the other hand, in LLM4DATA, LLMs are emerging as general-purpose engines for data management. We review recent advances in (i) data manipulation, including automatic data cleaning, integration, discovery; (ii) data analysis, covering reasoning over structured, semi-structured, and unstructured data, and (iii) system optimization (e.g., configuration tuning, query rewriting, anomaly diagnosis), powered by LLM techniques like retrieval-augmented prompting, task-specialized fine-tuning, and multi-agent collaboration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。