用Mamba模型提升文本检测的长距离依赖建模能力。
TextMamba: Scene Text Detector with Mamba
- 结合Mamba的选择机制与注意力层,增强长序列信息提取
- 在CTW1500等数据集上达到89.7%、89.2%、78.5%的F-measure
- 适合关注长文本或复杂场景文本检测的研究者
在场景文本检测中,基于Transformer的方法克服了传统卷积神经网络在全局特征提取上的局限。然而,多数方法直接使用原生Transformer注意力层作为编码器,未评估其跨域限制和固有缺陷:在建模长程依赖时易遗忘重要信息或关注无关表示。最近提出的状态空间模型Mamba通过线性复杂度选择机制,在长程依赖建模方面表现更优。为此,本文提出一种基于Mamba的新型场景文本检测器,将选择机制与注意力层融合,提升编码器从长序列中提取相关特征的能力。采用Top_k算法显式选择关键信息,减少无关信息干扰。同时设计双尺度前馈网络和嵌入金字塔增强模块,促进高维隐藏状态交互与多尺度特征融合。该方法在多个基准测试中达到当前最优或具有竞争力的性能,分别在CTW1500、TotalText、ICDAR19ArT上取得89.7%、89.2%、78.5%的F-measure。代码将公开。
原文摘要 · Abstract (English)
In scene text detection, Transformer-based methods have addressed the global feature extraction limitations inherent in traditional convolution neural network-based methods. However, most directly rely on native Transformer attention layers as encoders without evaluating their cross-domain limitations and inherent shortcomings: forgetting important information or focusing on irrelevant representations when modeling long-range dependencies for text detection. The recently proposed state space model Mamba has demonstrated better long-range dependencies modeling through a linear complexity selection mechanism. Therefore, we propose a novel scene text detector based on Mamba that integrates the selection mechanism with attention layers, enhancing the encoder's ability to extract relevant information from long sequences. We adopt the Top\_k algorithm to explicitly select key information and reduce the interference of irrelevant information in Mamba modeling. Additionally, we design a dual-scale feed-forward network and an embedding pyramid enhancement module to facilitate high-dimensional hidden state interactions and multi-scale feature fusion. Our method achieves state-of-the-art or competitive performance on various benchmarks, with F-measures of 89.7\%, 89.2\%, and 78.5\% on CTW1500, TotalText, and ICDAR19ArT, respectively. Codes will be available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。