arXiv:2602.08808cs.LG2026-02

构建网页如何生成数据集,评估并提升大模型步骤式推理能力。

How2Everything: Mining the Web for How-To Procedures to Evaluate and Improve LLMs

  • 从980K网页挖掘35.1万条真实步骤流程,覆盖14个主题。
  • 建立7000例评测集,发现模型性能随规模增长且早期预训练即有信号。
  • 用强化学习优化后,跨三模型性能提升超10分,不损害基准表现。

生成分步‘如何做’流程是大模型的关键能力:这类建议常被用于聊天机器人,且分步规划对复杂任务推理至关重要。然而,对真实世界任务中程序有效性进行大规模评估与改进仍具挑战且研究不足。为此,我们提出How2Everything,一个可扩展的框架,用于评估和改进目标导向的流程生成。该框架包含How2Mine,从980K网页中挖掘出35.1万条流程,覆盖14个主题,可轻松扩展至更大语料库;基于此构建了含7000例的平衡评测集How2Bench。为可靠评分,我们设计了How2Score,通过大模型裁判判断生成内容是否存在导致目标失败的关键错误。为实现低成本可复现评估,我们将前沿模型蒸馏为开源8B模型,与人工标注者达成80.5%一致率。How2Bench揭示了模型规模与训练阶段间的清晰缩放趋势,甚至在预训练早期就已出现信号。最后,使用How2Score作为奖励进行强化学习,使三个模型在How2Bench上的性能均提升超过10分,且无系统性退化,效果对表面记忆或格式合规性鲁棒。总体而言,How2Everything展示了预训练网络数据如何支持能力评估与改进的闭环。

原文摘要 · Abstract (English)

Generating step-by-step "how-to" procedures is a key LLM capability: how-to advice is commonly requested in chatbots, and step-by-step planning is critical for reasoning over complex tasks. Yet, measuring and improving procedural validity at scale on real-world tasks remains challenging and understudied. To address this, we introduce How2Everything, a scalable framework to evaluate and improve goal-conditioned procedure generation. Our framework includes How2Mine, which mines 351K procedures from 980K web pages across 14 topics and readily scales to larger corpora. From this pool we build How2Bench, a 7K-example evaluation set balanced across topics. To reliably score model outputs, we develop How2Score, an evaluation protocol that uses an LLM judge to detect whether a generation contains any critical failure that would prevent achieving the goal. For low-cost, reproducible evaluation, we distill a frontier model into an open 8B model, achieving 80.5% agreement with human annotators. How2Bench reveals clear scaling trends across model sizes and training stages, providing signal early in pretraining. Finally, RL using How2Score as a reward improves performance on How2Bench by >10 points across three models without systematic regressions on standard benchmarks, with gains robust to superficial source-document memorization or format compliance. Taken together, How2Everything shows how pretraining web data can support a closed loop of capability evaluation and improvement at scale.

流程生成大模型评估强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。