arXiv:2601.19923cs.CLcs.AI2026-01

提出自监督框架,精准评估大模型生成网页数据的结构准确性

Structure-BiEval: A Self-Supervised, Dual-Track Framework for Decoupling Structure and Content in LLM Evaluation for Web Information Systems

  • 用中间表示解耦结构与内容,实现无标注量化评估
  • 15个主流模型测试显示中小模型在结构生成上反而更优
  • 发现递归嵌套是跨规模模型的共性难题,适合网页系统开发者

随着大语言模型成为基于网络的自主代理和复杂网络信息系统的核心,其将自然语言准确转换为严谨结构化格式的能力至关重要,这直接影响网络API调用与数据交换。然而,评估此类网络原生数据的结构一致性仍面临挑战:传统文本指标无法捕捉半结构化网络数据的拓扑一致性,而人工评估成本过高。为此,我们提出Structure-BiEval,一种专为网络数据工程设计的自监督定量评估框架。通过利用确定性的中间表示,该框架有效解耦结构与内容,采用内容语义准确率与归一化树编辑距离作为精确度量标准。我们在两类网络结构拓扑上对15个前沿大模型进行实证基准测试,分别是层次化数据(网络后端数据)与表格数据(前端展示)。结果揭示出显著的结构性能差异,包括部分中等规模模型在网页数据格式化任务中意外优于更大模型。此外,研究还表明深度递归嵌套对各类参数规模的网络代理构成持续挑战。

原文摘要 · Abstract (English)

As Large Language Models (LLMs) evolve into the core of Web-based autonomous agents and complex Web Information Systems, their ability to faithfully translate natural language into rigorous structured formats has become paramount, as this capability is critical for Web API invocation and data exchange. However, evaluating this structural fidelity in Web-native payloads remains a challenge: traditional text metrics fail to capture topological consistency in semi-structured Web data, while manual evaluation is prohibitively costly. To address this, we propose Structure-BiEval, a novel self-supervised framework for quantitative, annotation-free assessment tailored for Web data engineering. By leveraging deterministic Intermediate Representations, our framework effectively decouples structure from content, utilizing Content Semantic Accuracy and Normalized Tree Edit Distance as precise metrics. We empirically benchmark 15 state-of-the-art LLMs across dual Web structural topologies, namely Hierarchical Data (Web backend payloads) and Tabular Data (Web frontend presentation). The results reveal substantial variability in structural performance, including cases where mid-sized models unexpectedly outperform larger counterparts in Web data formatting. Furthermore, our findings show that deep recursive nesting poses a consistent challenge for Web agents across varying parameter scales.

大模型评估网页数据结构生成自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。