Misanthrope不检测人形关键点,从源头保护定位隐私。
Misanthrope: A Privacy-Preserving Keypoint Detector

- 通过自蒸馏训练,让模型主动避开人体区域提取特征
- 在Phototourism数据集上9个场景中7个表现最佳,性能媲美顶尖模型
- 适合需要隐私保护的移动定位、SLAM等实时应用
图像匹配是同时定位与地图构建(SLAM)、视觉定位和运动结构(SfM)等应用的核心。然而,传统局部特征易受反演攻击,攻击者可从特征重建出敏感场景内容,尤其在边缘设备预计算特征后由远程服务器处理的分布式场景中风险更高。本文提出Misanthrope,一种通过自蒸馏训练的隐私保护关键点检测器,主动避免在人身上检测关键点——绝大多数定位场景中的主要隐私源,从而在源头防范反演攻击,而非事后模糊处理。我们验证了传统方法生成的反演图像可成功识别场景中的人,而Misanthrope能有效缓解此威胁。在保持与当前最优水平相当的图像匹配性能的同时,其在存在行人干扰的挑战性场景(如phototourism和in-the-wild odometry)中甚至表现更优。在Image Matching Challenge 2021 Phototourism测试集上,Misanthrope在9个场景中有7个排名第一。代码与评估脚本已开源:https://github.com/fratopa/misanthrope
原文摘要 · Abstract (English)
Image matching is a core component of applications such as Simultaneous Localization and Mapping (SLAM), Visual Localization, and Structure from Motion (SfM). However, the local image features central to this task are vulnerable to inversion attacks, which enable adversaries to reconstruct privacy-sensitive scene content from local features. These attacks pose a particular threat in distributed computing scenarios where the pre-computed features leave edge devices to be processed by remote servers. In this work, we introduce Misanthrope, a novel privacy-preserving keypoint detector trained through self-distillation to avoid detecting keypoints on people---a predominant source of privacy-sensitive content in most localization scenarios---thus mitigating inversion attacks at the source rather than through post-hoc obfuscation. We demonstrate how inverted images from traditional feature detection pipelines can be used to detect and re-identify people in the scene, while Misanthrope is able to mitigate these attacks. Furthermore, Misanthrope maintains image matching performance on par with the state of the art and even surpasses it in challenging settings where people act as distractors, such as phototourism and in-the-wild odometry. On the Image Matching Challenge 2021 Phototourism test set, Misanthrope is the top-performing sparse feature extractor in 7 out of 9 scenes. We make our model and its evaluation script available here: https://github.com/fratopa/misanthrope
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。