用结构化配置替代自由代码，让大模型生成的网络爬虫更稳定可靠。
Making Failure Safe: A Constrained, Verifiable Agent Framework for Open-Web Data Collection
- 将自然语言需求转为带约束的JSON配置，避免代码错误。
- 在80个任务中零执行阶段调用大模型，耗时最低且可重复运行。
- 适合需要定期、可验证数据采集的自动化场景。
大模型和智能体可从自然语言要求生成网页爬虫，但直接生成仍不可靠，常因依赖错误、选择器失效、字段映射不匹配及页面结构异构导致失败。本文提出一种受约束、可验证的智能体框架，将大模型输出从自由代码转为带类型的JSON收集器配置，结合六类收集器分类体系、模板与工具函数约束、静态Airflow DAG执行、基于规则的质量检查及结构化反馈修正机制。138项任务实验表明，该分类体系支持基于描述的需求类型识别；同时证实，稳定实例化需完成源、字段与执行三类约束。在80个独立源验证任务中，框架实现零执行阶段大模型调用，平均耗时最低，以适度的一次性质量为代价，提供可复用、确定性且可验证的执行路径，适用于重复调度的数据采集场景。
原文摘要 · Abstract (English)
LLMs and agents can generate web scrapers from natural-language requirements, but direct generation remains unreliable because of dependency errors, broken selectors, schema mismatches, and heterogeneous page structures. We propose a constrained, verifiable agent framework that shifts LLM output from free-form code to typed JSON collector configurations, combining a six-type collector taxonomy, template and utility-function constraints, static Airflow DAG execution, rule-based quality checking, and structured feedback correction. Experiments on 138 tasks show that the taxonomy supports description-based requirement typing, while confirming that stable instantiation requires completing source, field, and execution constraints beyond the initial description. On 80 independently source-verified tasks, the framework runs with zero execution-stage LLM tokens and the lowest average wall-clock time, trading moderate one-shot quality for a reusable, deterministic, and verifiable execution path suited to repeated scheduled collection. These results position the framework as a reusable, low-cost, and verifiable execution path for repeated open-web data collection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。