通过智能剪枝与可复用脚本生成，提升网页数据提取效率与准确率。
Co-Scraper: query-aware DOM Pruning and Reusable Scraper Synthesis for Lightweight Web Data Extraction
- 基于查询感知的DOM剪枝，精准保留关键节点。
- 在SWDE测试集上达到94.78%的F1值和90.39%的复用率。
- 适合需要高效、可复用爬虫的网页数据采集场景。
网页内容丰富且结构多样，自动化信息抽取需求迫切。本文提出Co-Scraper，一种两阶段框架，能够处理长HTML文档的层级复杂性。通过集成查询感知的DOM剪枝机制与稳定的抽取策略归纳，Co-Scraper利用微调后的Qwen3-8B模型将网页内容转化为可执行的程序化包装器。在SWDE测试集上，Co-Scraper实现94.78%的F1分数和90.39%的复用成功率，显著提升数据提取的准确性与鲁棒性，为网页数据获取提供高效解决方案。
原文摘要 · Abstract (English)
The abundant and heterogeneous nature of web content necessitates automated information extraction, and generating scrapers that can be reused across similar web pages offers an effective solution for scalable data extraction. In this work, we propose Co-Scraper, a two-stage framework capable of handling the hierarchical complexity of long HTML documents. By integrating a query-aware DOM pruning mechanism with stable extraction strategy induction, Co-Scraper can effectively transforms web content into executable programmatic wrappers using a fine-tuned Qwen3-8B model. On the test set of SWDE, Co-Scraper achieves state-of-the-art performance with an F1 score of 94.78% and a reuse success rate of 90.39%. This framework significantly enhances the accuracy and resilience of data extraction, providing a highly efficient approach for web data acquisition tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。