仅用少量手写样本实现推理时自适应，无需参数更新
Few-shot Writer Adaptation via Multimodal In-Context Learning
- 基于多模态上下文学习，推理时仅需几例即可适配新作者
- IAM和RIMES数据集上字符错误率分别低至3.92%和2.34%
- 模型仅800万参数，适合资源受限场景的快速部署
当前最先进的手写文本识别（HTR）模型在标准基准上表现良好，但在训练数据中代表性不足的特殊书写风格上常表现不佳。为应对未见且异常的书写者，写作适配技术可个性化调整HTR模型以匹配个体笔迹。现有主流方法需离线微调或推理时参数更新，均涉及梯度计算与反向传播，增加计算开销并需精细调参。本文提出一种受多模态上下文学习启发的新型上下文驱动HTR框架，仅需目标书写者的少量样本即可在推理时完成适配，无需任何参数更新。我们进一步研究了上下文长度影响，设计出一个仅800万参数的CNN-Transformer模型，支持少样本上下文适配，并验证结合上下文驱动与标准OCR训练策略可带来互补性提升。在IAM和RIMES数据集上的实验表明，该方法分别取得3.92%和2.34%的字符错误率，超越所有无需推理时参数更新的无监督HTR模型。
原文摘要 · Abstract (English)
While state-of-the-art Handwritten Text Recognition (HTR) models perform well on standard benchmarks, they frequently struggle with writers exhibiting highly specific styles that are underrepresented in the training data. To handle unseen and atypical writers, writer adaptation techniques personalize HTR models to individual handwriting styles. Leading writer adaptation methods require either offline fine-tuning or parameter updates at inference time, both involving gradient computation and backpropagation, which increase computational costs and demand careful hyperparameter tuning. In this work, we propose a novel context-driven HTR framework3 inspired by multimodal in-context learning, enabling inference-time writer adaptation using only a few examples from the target writer without any parameter updates. We further demonstrate the impact of context length, design a compact 8M-parameter CNN-Transformer that enables few-shot in-context adaptation, and show that combining context-driven and standard OCR training strategies leads to complementary improvements. Experiments on IAM and RIMES validate our approach with Character Error Rates of 3.92% and 2.34%, respectively, surpassing all writer-independent HTR models without requiring any parameter updates at inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。