让自然语言直接生成数据处理流水线，降低非专业人士使用门槛
Text-to-Pipeline: Bridging Natural Language and Data Preparation Pipelines
- 用自然语言指令自动构建数据处理流水线，结合真实场景设计任务
- 现有大模型在多步骤逻辑和参数语义理解上表现不佳，存在明显缺陷
- 适合研究自动化数据处理、智能代理系统方向的开发者和研究者
数据准备（DP）将原始数据转化为下游应用可用的形式，通常通过组合操作形成可执行的流水线。构建这些流水线耗时且需要高超编程技能，对非专家构成显著障碍。为降低这一门槛，我们提出文本到流水线（Text-to-Pipeline）新任务，并构建了大规模基准PARROT，用于系统评估。PARROT基于生产环境流水线挖掘转换模式，在23,009个真实表格上实例化，生成约18,000个任务，涵盖16种核心操作符。在该基准上的实证评估发现，前沿大模型不仅在多步组合逻辑上表现差，还存在语义参数定位失败的问题。为此我们提出了Pipeline-Agent，一种具备执行感知能力的迭代反思智能体，虽达到当前最优性能，但仍存在显著差距，凸显该任务的深层挑战。PARROT为下一代自主数据准备智能体系统提供了关键的大规模测试平台。
原文摘要 · Abstract (English)
Data preparation (DP) transforms raw data into a form suitable for downstream applications, typically by composing operations into executable pipelines. Building such pipelines is time-consuming and requires sophisticated programming skills, posing a significant barrier for non-experts. To lower this barrier, we introduce Text-to-Pipeline, a new task that translates NL data preparation instructions into DP pipelines, and PARROT, a large-scale benchmark to support systematic evaluation. To ensure realistic DP scenarios, PARROT is built by mining transformation patterns from production pipelines and instantiating them on 23,009 real-world tables, resulting in ~18,000 tasks spanning 16 core operators. Our empirical evaluation on PARROT reveals a critical failure mode in cutting-edge LLMs: they struggle not only with multi-step compositional logic but also with semantic parameter grounding. We thus establish a strong baseline with Pipeline-Agent, an execution-aware agent that iteratively reflects on intermediate states. While it achieves state-of-the-art performance, a significant gap remains, underscoring the deep, unsolved challenges for PARROT. It provides the essential, large-scale testbed for developing and evaluating the next generation of autonomous data preparation agentic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。