发现模型仓库中可被利用的恶意配置，提出检测工具防止代码劫持。
A Rusty Link in the AI Supply Chain: Detecting Evil Configurations in Model Repositories
- 用大模型分析配置文件与运行时代码的上下文关系。
- 在数千个仓库中检出可疑配置，误报率低、准确率高。
- 适合关注AI供应链安全的研究者与平台维护者。
大型语言模型的快速发展推动了代码生成、视频编辑和文本生成等多样化AI应用的兴起；然而，像Hugging Face这样由公众贡献预训练模型及其配置文件的AI供应链平台面临严重安全挑战。配置文件本用于设置模型参数和初始状态，却可能被恶意利用以执行未经授权的代码。现有研究大多忽视了配置文件的安全风险，而聚焦于模型本身。本文首次对Hugging Face上的恶意配置进行系统性研究，识别出三类攻击场景：文件操作、网站操作和仓库操作，揭示其固有风险。为此，我们提出CONFIGSCAN，一种基于大模型的检测工具，通过分析配置文件与其关联的运行时代码及关键库的上下文关系，有效识别可疑元素，实现低误报率和高准确率。大规模评估发现数千个可疑仓库和配置文件，凸显了提升AI模型托管平台安全验证的紧迫性。
原文摘要 · Abstract (English)
Recent advancements in large language models (LLMs) have spurred the development of diverse AI applications from code generation and video editing to text generation; however, AI supply chains such as Hugging Face, which host pretrained models and their associated configuration files contributed by the public, face significant security challenges; in particular, configuration files originally intended to set up models by specifying parameters and initial settings can be exploited to execute unauthorized code, yet research has largely overlooked their security compared to that of the models themselves; in this work, we present the first comprehensive study of malicious configurations on Hugging Face, identifying three attack scenarios (file, website, and repository operations) that expose inherent risks; to address these threats, we introduce CONFIGSCAN, an LLM-based tool that analyzes configuration files in the context of their associated runtime code and critical libraries, effectively detecting suspicious elements with low false positive rates and high accuracy; our extensive evaluation uncovers thousands of suspicious repositories and configuration files, underscoring the urgent need for enhanced security validation in AI model hosting platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。