提出多模态记忆库缓解异常检测中模型遗忘问题
Multimodal Task Representation Memory Bank vs. Catastrophic Forgetting in Anomaly Detection
- 用关键提示引导BERT与ViT跨模态交互，提升特征融合
- 基于结构对比学习生成精准分割掩码，保持特征一致性
- 在MVtec AD和VisA上实现0.921平均准确率，遗忘率最低
无监督连续异常检测（UCAD）在多任务表征学习中面临表征不完整和灾难性遗忘的挑战。由于缺乏先验信息，现有方法难以有效区分冗余与互补的多模态特征。为此，我们提出多模态任务表征记忆库（MTRMB），包含两项关键技术：一是关键提示多模态知识（KPMK）机制，利用简洁的关键提示引导BERT与ViT之间的跨模态特征交互；二是基于结构的对比学习（RSCL），结合Grounding DINO与SAM生成精确分割掩码，使相同结构区域的特征更接近，不同结构区域的特征更分离。在MVtec AD和VisA数据集上的实验表明，MTRMB在最低遗忘率下达到0.921的平均检测准确率，显著优于现有最先进方法。代码计划开源。
原文摘要 · Abstract (English)
Unsupervised Continuous Anomaly Detection (UCAD) faces significant challenges in multi-task representation learning, with existing methods suffering from incomplete representation and catastrophic forgetting. Unlike supervised models, unsupervised scenarios lack prior information, making it difficult to effectively distinguish redundant and complementary multimodal features. To address this, we propose the Multimodal Task Representation Memory Bank (MTRMB) method through two key technical innovations: A Key-Prompt-Multimodal Knowledge (KPMK) mechanism that uses concise key prompts to guide cross-modal feature interaction between BERT and ViT. Refined Structure-based Contrastive Learning (RSCL) leveraging Grounding DINO and SAM to generate precise segmentation masks, pulling features of the same structural region closer while pushing different structural regions apart. Experiments on MVtec AD and VisA datasets demonstrate MTRMB's superiority, achieving an average detection accuracy of 0.921 at the lowest forgetting rate, significantly outperforming state-of-the-art methods. We plan to open source on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。