用智能模型统一不同数据集的标注标准,提升文档布局识别准确率
Improving Layout Representation Learning Across Inconsistently Annotated Datasets via Agentic Harmonization
- 用视觉语言模型自动调和不同数据集的类别语义与框粒度差异
- 在两个共享8类对应关系的数据集上,检测F-score提升至0.883,表格识别准确率达0.814
- 适合处理标注不一致的多源文档数据,尤其对布局理解任务有显著帮助
将物体检测模型在合并数据集上微调的前提是标注一致性,但不同数据集常对语义相同的类别采用不同的空间定义。本文提出一种基于智能体的标签调和流程,利用视觉语言模型在训练前统一异构数据源的类别语义与边界框粒度。以文档布局检测为例,该任务在不同语料库中标注标准差异大。未经调和的混合数据微调使预训练的RT-DETRv2检测器性能下降:在SCORE-Bench上,表格TEDS从0.800降至0.750。针对两个16类与10类分类体系仅共享8个直接对应关系的数据集,调和后在内容保真度、表格结构和空间一致性上均获提升:检测F-score从0.860升至0.883,表格TEDS达0.814,平均框重叠率从0.043降至0.016。表示分析进一步表明,调和训练产生的解码后嵌入更紧凑且可分,证明标注不一致会扭曲特征空间,而提前解决此问题可恢复表征结构。
原文摘要 · Abstract (English)
Fine-tuning object detection (OD) models on combined datasets assumes annotation compatibility, yet datasets often encode conflicting spatial definitions for semantically equivalent categories. We propose an agentic label harmonization workflow that uses a vision-language model to reconcile both category semantics and bounding box granularity across heterogeneous sources before training. We evaluate on document layout detection as a challenging case study, where annotation standards vary widely across corpora. Without harmonization, naïve mixed-dataset fine-tuning degrades a pretrained RT-DETRv2 detector: on SCORE-Bench, which measures how accurately the full document conversion pipeline reproduces ground-truth structure, table TEDS drops from 0.800 to 0.750. Applied to two corpora whose 16 and 10 category taxonomies share only 8 direct correspondences, harmonization yields consistent gains across content fidelity, table structure, and spatial consistency: detection F-score improves from 0.860 to 0.883, table TEDS improves to 0.814, and mean bounding box overlap drops from 0.043 to 0.016. Representation analysis further shows that harmonized training produces more compact and separable post-decoder embeddings, confirming that annotation inconsistency distorts the learned feature space and that resolving it before training restores representation structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。