arXiv:2603.03790cs.CLcs.AI2026-03被引 2

用结构化思维提升模型理解复杂文本能力,效果显著。

T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning

  • 提出结构化思维提示法,引导模型构建中间文本结构
  • 多任务平均准确率提升5.7%,顶尖模型节点准确率达58.1%
  • 适合研究模型推理能力与文本理解的开发者使用

人类处理复杂阅读任务时会标记重点、推断关系并组织信息以辅助理解。大语言模型是否也能通过显式文本结构提升处理能力?本文首先提出结构化思维(SoT)提示方法,引导模型构建中间文本结构,在八个任务和三个模型家族中均实现性能提升。基于此,我们构建了首个评估文本到结构能力的基准T2S-Bench,涵盖6个科学领域、32种结构类型,共1.8K样本,确保准确性和公平性。对45个主流模型的评估显示,多跳推理任务平均准确率仅52.1%,即使最先进模型在端到端提取中也仅达58.1%节点准确率。在Qwen2.5-7B-Instruct上,仅使用SoT即带来平均+5.7%提升,结合T2S-Bench微调后进一步增至+8.6%。结果表明显式文本结构具有重要价值,且SoT与T2S-Bench可协同增效。数据集与评测代码已公开。

原文摘要 · Abstract (English)

Think about how human handles complex reading tasks: marking key points, inferring their relationships, and structuring information to guide understanding and responses. Likewise, can a large language model benefit from text structure to enhance text-processing performance? To explore it, in this work, we first introduce Structure of Thought (SoT), a prompting technique that explicitly guides models to construct intermediate text structures, consistently boosting performance across eight tasks and three model families. Building upon this insight, we present T2S-Bench, the first benchmark designed to evaluate and improve text-to-structure capabilities of models. T2S-Bench includes 1.8K samples across 6 scientific domains and 32 structural types, rigorously constructed to ensure accuracy, fairness, and quality. Evaluation on 45 mainstream models reveals substantial improvement potential: the average accuracy on the multi-hop reasoning task is only 52.1%, and even the most advanced model achieves 58.1% node accuracy in end-to-end extraction. Furthermore, on Qwen2.5-7B-Instruct, SoT alone yields an average +5.7% improvement across eight diverse text-processing tasks, and fine-tuning on T2S-Bench further increases this gain to +8.6%. These results highlight the value of explicit text structuring and the complementary contributions of SoT and T2S-Bench. Dataset and eval code have been released at https://t2s-bench.github.io/T2S-Bench-Page/.

文本理解提示工程推理增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。