arXiv:2604.00382cs.CVeess.SP2026-04中稿 · the 24th ACM/IEEE …

用视觉信息辅助毫米波雷达,让系统更准识别异常。

mmAnomaly: Leveraging Visual Context for Robust Anomaly Detection in the Non-Visual World with mmWave Radar

  • 融合毫米波与RGBD数据,利用视觉上下文预测正常信号
  • 在三种场景下达94%的F1分数,定位误差小于1米
  • 适合安防、救援等需穿墙感知的隐私敏感场景

毫米波雷达可在非可视场景(如穿衣物或墙体)中实现人体感知,但信号易受材料特性、杂波和多径干扰影响,产生复杂非高斯畸变,导致现有方法缺乏上下文意识,将正常信号变化误判为异常。我们提出mmAnomaly,一种结合毫米波雷达与RGBD输入的多模态异常检测框架。系统通过快速的ResNet分类器提取场景几何与材质等语义线索,并采用条件潜空间扩散模型生成给定视觉上下文下的预期毫米波频谱。双输入对比模块通过比较真实与生成频谱的空间差异,实现异常定位。我们在两个多模态数据集上评估了该系统在三种应用中的表现:隐匿武器定位、穿墙入侵者定位及穿墙跌倒定位。结果表明,系统在各类条件下均达到最高94% F1分数,定位误差低于1米,展现出对衣物遮挡、遮挡物及复杂环境的强泛化能力。这证明mmAnomaly是一种准确且可解释的上下文感知毫米波异常检测框架。

原文摘要 · Abstract (English)

mmWave radar enables human sensing in non-visual scenarios-e.g., through clothing or certain types of walls-where traditional cameras fail due to occlusion or privacy limitations. However, robust anomaly detection with mmWave remains challenging, as signal reflections are influenced by material properties, clutter, and multipath interference, producing complex, non-Gaussian distortions. Existing methods lack contextual awareness and misclassify benign signal variations as anomalies. We present mmAnomaly, a multi-modal anomaly detection framework that combines mmWave radar with RGBD input to incorporate visual context. Our system extracts semantic cues-such as scene geometry and material properties-using a fast ResNet-based classifier, and uses a conditional latent diffusion model to synthesize the expected mmWave spectrum for the given visual context. A dual-input comparison module then identifies spatial deviations between real and generated spectra to localize anomalies. We evaluate mmAnomaly on two multi-modal datasets across three applications: concealed weapon localization, through-wall intruder localization, and through-wall fall localization. The system achieves up to 94% F1 score and sub-meter localization error, demonstrating robust generalization across clothing, occlusions, and cluttered environments. These results establish mmAnomaly as an accurate and interpretable framework for context-aware anomaly detection in mmWave sensing.

毫米波雷达异常检测多模态融合安防应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。