用分面锚点与动态共识提升文本搜索真实场景异常行人效果
FaLCon: Facet-Anchored Retrieval with Late Consensus for Sim2Real Text-Based Person Anomaly Search

- 分面锚点引导粗到精检索,保留全局匹配能力
- 多模块验证将准确率提至mAP@10 95.41%
- 适合需要高精度细粒度图像搜索的应用场景
基于文本的行人异常搜索需从真实世界行人图像中检索符合自然语言描述的结果,而模型主要在合成数据上训练。此跨域(Sim2Real)场景极具挑战:视觉相似候选者仅在动作、交互或外观属性上有细微差异,且对全库使用多模态大模型计算开销巨大。本文提出一种锚定约束的粗到精检索框架,结合全局语义匹配与细粒度验证。每个查询通过原始描述、结构化拼接句及多个语义分面表示;异构视觉-语言检索器通过每查询得分校准与软声明感知融合集成。完整与拼接描述作为锚点以保持召回率,外观、动作、物体分面提供有界修正证据。候选集经判别式Qwen3重排序器及两个互补的语义验证模块——异常感知填空与多智能体证据推理进一步优化。最后,不确定性门控共识模块在模糊查询下自适应加权三个专家。在PAB基准测试中,软声明感知检索达86.44% mAP@10,整体框架进一步提升至95.41% mAP@10、94.44% R@1、99.09% R@5。结果表明,在保持强全局检索的同时,将昂贵语义推理限制于小候选池是细粒度跨域行人异常搜索的有效策略。代码将开源。
原文摘要 · Abstract (English)
Text-based person anomaly search requires retrieving real-world pedestrian images from detailed natural-language descriptions using models trained primarily on synthetic data. This Sim2Real setting is particularly challenging because visually similar candidates may differ only in subtle actions, object interactions, or appearance attributes, while applying multimodal large language models to the entire gallery is computationally expensive. We propose an anchor-constrained coarse-to-fine retrieval framework that combines global semantic matching with fine-grained verification. First, each query is represented by its original caption, a structured concatenation, and several semantic facets. Heterogeneous vision-language retrievers are then integrated through robust per-query score calibration and soft claim-aware fusion. Full and concatenated captions serve as anchors to preserve candidate recall, whereas appearance, action, and object facets provide bounded corrective evidence. The resulting candidate pool is further refined by a discriminative Qwen3 reranker and two complementary semantic verification modules based on anomaly-aware cloze completion and multi-agent evidence reasoning. Finally, an uncertainty-gated consensus module adaptively reweights the three experts on ambiguous queries. Experiments on the PAB benchmark show that the proposed soft claim-aware retrieval achieves 86.44% mAP@10, substantially outperforming individual retrieval backbones. The complete framework further improves performance to 95.41% mAP@10, 94.44% R@1, and 99.09% R@5. These results demonstrate that preserving strong global retrieval while restricting expensive semantic reasoning to a small candidate pool is effective for fine-grained Sim2Real person anomaly search. Our code will be available on Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。