1.5亿参数小模型高效提取网页内容，输出整洁的Markdown或JSON。
ReaderLM-v2: Small Language Model for HTML to Markdown and JSON
- 三阶段数据合成生成高质量训练数据，提升内容提取能力。
- 在超长文档（>10万词元）上比GPT-4o高15-20%准确率。
- 适合需要轻量级网页结构化处理的开发者与LLM应用者。
我们提出ReaderLM-v2，一个仅15亿参数的紧凑语言模型，专为高效网页内容提取设计。该模型可处理长达512K词元的文档，将混乱的HTML转换为清晰的Markdown或JSON格式，准确率高，是大模型落地的理想工具。其性能得益于两大创新：(1) 三阶段数据合成流水线，通过迭代撰写、优化和评判生成高质量、多样化的训练数据；(2) 统一训练框架，结合连续预训练与多目标优化。大量评估表明，ReaderLM-v2在精心设计的基准测试中优于GPT-4o-2024-08-06及其他更大模型15-20%，尤其在超过10万词元的文档上表现突出，同时计算开销显著更低。
原文摘要 · Abstract (English)
We present ReaderLM-v2, a compact 1.5 billion parameter language model designed for efficient web content extraction. Our model processes documents up to 512K tokens, transforming messy HTML into clean Markdown or JSON formats with high accuracy -- making it an ideal tool for grounding large language models. The model's effectiveness results from two key innovations: (1) a three-stage data synthesis pipeline that generates high quality, diverse training data by iteratively drafting, refining, and critiquing web content extraction; and (2) a unified training framework combining continuous pre-training with multi-objective optimization. Intensive evaluation demonstrates that ReaderLM-v2 outperforms GPT-4o-2024-08-06 and other larger models by 15-20\% on carefully curated benchmarks, particularly excelling at documents exceeding 100K tokens, while maintaining significantly lower computational requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。