arXiv:2603.12718cs.CV2026-03被引 1

提出可分解的COTe评分框架,更精准评估文档版面分析模型性能。

The COTe score: A decomposable framework for evaluating Document Layout Analysis models

  • 引入语义单元结构(SSU)和可拆解的COTe评分体系
  • 相比传统指标,揭示模型在语义边界越界等失败模式
  • 支持低门槛使用,适合文档分析研究者与开发者

文档版面分析(DLA)是将页面解析为有意义元素的过程,常依赖机器学习模型。传统评估指标如IoU、F1或mAP源自2D图像检测,不适用于印刷媒体的原生2D特性,易导致性能解读偏差。为此,本文提出结构语义单元(SSU)关系标注方法,将关注点从物理布局转向语义结构;并构建覆盖、重叠、侵入与冗余(COTe)评分体系,实现页面解析质量的可分解评估。通过案例研究及在3个DLA数据集上对5种常见模型的评测,验证了COTe比传统指标更具信息量,能揭示模型在语义边界越界或重复解析同一区域等差异性失败模式。同时,COTe将解释-性能差距降低达76%(相对F1)。值得注意的是,即使无显式SSU标注,其粒度鲁棒性仍显著,降低了使用门槛。最后,公开了带SSU标注的数据集及Python工具库,支持在实际项目中应用COTe。

原文摘要 · Abstract (English)

Document Layout analysis (DLA), is the process by which a page is parsed into meaningful elements, often using machine learning models. Typically, the quality of a model is judged using general object detection metrics such as IoU, F1 or mAP. However, these metrics are designed for images that are 2D projections of 3D space, not for the natively 2D imagery of printed media. This discrepancy can result in misleading or uninformative interpretation of model performance by the metrics. To encourage more robust, comparable, and nuanced DLA, we introduce: The Structural Semantic Unit (SSU) a relational labelling approach that shifts the focus from the physical to the semantic structure of the content; and the Coverage, Overlap, Trespass, and Excess (COTe) score, a decomposable metric for measuring page parsing quality. We demonstrate the value of these methods through case studies and by evaluating 5 common DLA models on 3 DLA datasets. We show that the COTe score is more informative than traditional metrics and reveals distinct failure modes across models, such as breaching semantic boundaries or repeatedly parsing the same region. In addition, the COTe score reduces the interpretation-performance gap by up to 76% relative to the F1. Notably, we find that the COTe's granularity robustness largely holds even without explicit SSU labelling, lowering the barriers to entry for using the system. Finally, we release an SSU labelled dataset and a Python library for applying COTe in DLA projects.

文档分析评估指标语义结构可分解评分

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。