arXiv:2602.21225cs.CLcs.AI2026-02

通过渐进式数据调度,训练效率提升33%,适合资源有限的模型。

Architecture-Agnostic Curriculum Learning for Document Understanding: Empirical Evidence from Text-Only and Multimodal

  • 按33%→67%→100%逐步增加数据量,降低训练时间
  • 在文本模型上提升F1达+0.023,计算量不变下仍有效
  • 多模态模型无明显收益,说明效果依赖模型能力与任务复杂度

我们研究了渐进式数据调度(课程学习)是否能在不同架构的文档理解模型中带来一致的效率提升。在FUNSD和CORD基准上,对仅文本的BERT(110M参数)和多模态的LayoutLMv3(126M参数)进行评估,发现该调度使实际训练时间减少约33%,相当于有效训练轮次从6.67降至10.0。为排除计算量影响,引入匹配计算基线(Standard-7)。在FUNSD上,课程学习显著优于基线(ΔF1 = +0.023,p=0.022,d_z=3.83),证明容量受限模型存在真实调度收益;而LayoutLMv3未见类似提升(p=0.621),因其多模态表示已具备足够归纳偏置。在CORD上,所有条件最终均达到≥0.947的F1分数,表明存在性能上限。消融实验显示,效率提升源于数据量减少而非顺序设计。结果表明,渐进调度是跨模型家族可靠的降耗策略,其优势取决于模型容量与任务复杂度的交互作用。

原文摘要 · Abstract (English)

We investigate whether progressive data scheduling -- a curriculum learning strategy that incrementally increases training data exposure (33\%$\rightarrow$67\%$\rightarrow$100\%) -- yields consistent efficiency gains across architecturally distinct document understanding models. By evaluating BERT (text-only, 110M parameters) and LayoutLMv3 (multimodal, 126M parameters) on the FUNSD and CORD benchmarks, we establish that this schedule reduces wall-clock training time by approximately 33\%, commensurate with the reduction from 6.67 to 10.0 effective epoch-equivalents of data. To isolate curriculum effects from compute reduction, we introduce matched-compute baselines (Standard-7) that control for total gradient updates. On the FUNSD dataset, the curriculum significantly outperforms the matched-compute baseline for BERT ($Δ$F1 = +0.023, $p=0.022$, $d_z=3.83$), constituting evidence for a genuine scheduling benefit in capacity-constrained models. In contrast, no analogous benefit is observed for LayoutLMv3 ($p=0.621$), whose multimodal representations provide sufficient inductive bias. On the CORD dataset, all conditions converge to equivalent F1 scores ($\geq$0.947) irrespective of scheduling, indicating a performance ceiling. Schedule ablations comparing progressive, two-phase, reverse, and random pacing confirm that the efficiency gain derives from reduced data volume rather than ordering. Taken together, these findings demonstrate that progressive scheduling is a reliable compute-reduction strategy across model families, with curriculum-specific benefits contingent on the interaction between model capacity and task complexity.

课程学习文档理解训练优化模型效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。