提出混合全局与局部特征的文本行人异常检索框架，性能显著提升。
Hybrid, Unified and Iterative: A Novel Framework for Text-based Person Anomaly Retrieval
- 融合细粒度与粗粒度特征，设计局部-全局混合视角模块。
- 在PAB数据集上实现R@1提升9.70%，达当前最佳性能。
- 适合关注跨模态检索与异常检测的研究者参考。
文本驱动的行人异常检索是一项挑战性任务，现有方法多依赖复杂深度学习技术。为优化模型以提取更精细特征，本文提出局部-全局混合视角（LHP）模块，结合视觉语言模型，探索细粒度与粗粒度特征的协同作用。进一步设计统一图像-文本（UIT）模型，集成图像-文本对比（ITC）、匹配（ITM）、掩码语言（MLM）与掩码图像（MIM）损失。提出一种迭代集成策略，区别于传统并行集成，通过逐步融合提升效果。同时引入基于LHP指导的新型特征选择算法，增强模型表现。大量实验表明，该方法在PAB数据集上达到当前最优性能，相较于此前工作，R@1提升9.70%，R@5提升1.77%，R@10提升1.01%。
原文摘要 · Abstract (English)
Text-based person anomaly retrieval has emerged as a challenging task, with most existing approaches relying on complex deep-learning techniques. This raises a research question: How can the model be optimized to achieve greater fine-grained features? To address this, we propose a Local-Global Hybrid Perspective (LHP) module integrated with a Vision-Language Model (VLM), designed to explore the effectiveness of incorporating both fine-grained features alongside coarse-grained features. Additionally, we investigate a Unified Image-Text (UIT) model that combines multiple objective loss functions, including Image-Text Contrastive (ITC), Image-Text Matching (ITM), Masked Language Modeling (MLM), and Masked Image Modeling (MIM) loss. Beyond this, we propose a novel iterative ensemble strategy, by combining iteratively instead of using model results simultaneously like other ensemble methods. To take advantage of the superior performance of the LHP model, we introduce a novel feature selection algorithm based on its guidance, which helps improve the model's performance. Extensive experiments demonstrate the effectiveness of our method in achieving state-of-the-art (SOTA) performance on PAB dataset, compared with previous work, with a 9.70\% improvement in R@1, 1.77\% improvement in R@5, and 1.01\% improvement in R@10.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。