提出三阶段框架,系统挖掘100种模型指纹方案。
Queries, Representation & Detection: The Next 100 Model Fingerprinting Schemes
- 将指纹生成拆解为查询、表征、检测三部分,构建可组合方法论。
- 简单基线性能媲美复杂方案,揭示现有评估存在偏差。
- 开源工具箱,助力新指纹与更严谨基准的开发。
机器学习模型在实际部署中是组织的重大投资,因此被竞争对手窃取的风险需被重视。近年来已有诸多模型盗用检测方案提出,但它们基于隐含且不一致的数据与模型访问假设,导致难以有效比较。我们的评估表明,所提出的简单基线性能与现有最先进指纹方案相当,而后者却复杂得多。为揭示这一现象,本文提出系统化方法,将模型指纹分为三个核心组件——查询(Query)、表征(Representation)与检测(Detection),即QuRD,从而识别出约100种此前未探索的组合,并获得性能洞察。最后,我们引入一组指标,用于比较和指导更具代表性的模型盗用检测基准建设。研究揭示了需要更挑战性的基准及与基线的合理对比。为推动新指纹方案与基准的开发,我们开源了指纹工具箱。
原文摘要 · Abstract (English)
The deployment of machine learning models in operational contexts represents a significant investment for any organisation. Consequently, the risk of these models being misappropriated by competitors needs to be addressed. In recent years, numerous proposals have been put forth to detect instances of model stealing. However, these proposals operate under implicit and disparate data and model access assumptions; as a consequence, it remains unclear how they can be effectively compared to one another. Our evaluation shows that a simple baseline that we introduce performs on par with existing state-of-the-art fingerprints, which, on the other hand, are much more complex. To uncover the reasons behind this intriguing result, this paper introduces a systematic approach to both the creation of model fingerprinting schemes and their evaluation benchmarks. By dividing model fingerprinting into three core components -- Query, Representation and Detection (QuRD) -- we are able to identify $\sim100$ previously unexplored QuRD combinations and gain insights into their performance. Finally, we introduce a set of metrics to compare and guide the creation of more representative model stealing detection benchmarks. Our approach reveals the need for more challenging benchmarks and a sound comparison with baselines. To foster the creation of new fingerprinting schemes and benchmarks, we open-source our fingerprinting toolbox.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。