arXiv:2608.25358cs.AI2026-08

区分结构与内容错误,提升大模型生成表格和JSON的准确性

Where vs What: Decomposing Structural and Content Failures in LLM-Generated Structured Outputs

论文配图:Where vs What: Decomposing Structural and Content Failures in LLM-Generated Structured Outputs
图 1 · 摘自论文原文
  • 将结构错误与内容错误分离评估,精准定位生成问题
  • 复杂任务下结构准确率下降更快,最高达74%值位置错误
  • 设计可验证奖励机制,让模型学会正确安排数据位置

结构化输出如JSON和表格在现代大模型系统中至关重要,但生成错误常被笼统评估,混淆了位置错误(正确值放错位置)与内容错误(错误值在正确位置)。本文提出结构-内容分解(SCD)框架,独立衡量结构保真度与内容准确性。在六个模型(7B至前沿模型)上测试嵌套JSON与表格任务,发现随着复杂度增加,结构保真度衰减早且剧烈。在最高复杂度下,即使具备推理能力的DeepSeek-V4-Flash仍有35%的值被错位,Qwen2.5-7B高达74%。消融实验表明该现象源于依赖语义捷径而非对输出拓扑的理解。基于此,提出SA-RLVR,将SCD指标转化为可验证奖励用于强化学习(GRPO)。该方法成功提升JSON值位置准确率(VPA)从26%至63%,并泛化到未见模式;同时在表格领域持续提升VPA,证明结构感知奖励可直接改善多领域结构定位能力。

原文摘要 · Abstract (English)

Structured outputs such as JSON and tables are central to modern LLM-based systems, yet generation failures are evaluated monolithically, conflating two distinct error modes: placement errors (correct values at wrong positions) and value errors (wrong values at intended positions). We introduce Structure-Content Decomposition (SCD), a framework that independently measures structural fidelity and content accuracy. Applying SCD to nested JSON and table tasks across six models (7B to frontier), we uncover a consistent phenomenon: structural fidelity degrades earlier and more sharply than content accuracy as complexity increases. At the highest complexity, even DeepSeek-V4-Flash (with reasoning) misplaces 35% of recalled values, while Qwen2.5-7B misplaces 74%. Controlled ablations suggest that this pattern is associated with reliance on semantic shortcuts rather than topological understanding of output structure. Based on these findings, we propose SA-RLVR, converting SCD metrics into verifiable rewards for reinforcement learning via GRPO. SA-RLVR successfully optimizes structural addressing across distinct topologies: it lifts JSON Value Placement Accuracy (VPA) from 26% to 63% while generalizing to held-out schemas; moreover, it consistently drives VPA improvements in the table domain, demonstrating that structure-aware rewards can directly enhance multi-domain structural positioning.

大模型生成结构化输出错误分解强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。