分析76万模型与17.5万数据集的文档与许可证问题,揭示机器学习供应链隐患。
An Empirical Analysis of Machine Learning Model and Dataset Documentation, Supply Chain, and Licensing Challenges on Hugging Face
- 剖析Hugging Face上76万模型和17.5万数据集的文档现状,识别关键缺陷
- 发现模型与数据集之间存在大量依赖关系,且许可证信息不一致
- 适合关注AI合规、安全与可复现性的研究者与开发者参考
过去十年间,机器学习(ML)组件在各类软件系统中广泛应用,涵盖自然语言处理与计算机视觉等众多领域。这些组件从简单神经网络到资源密集型大语言模型不等。然而,尽管广泛采用,关于其生产供应链关系的信息仍极为有限,可能影响合规性与安全性。本文对Hugging Face平台上的760,460个模型和175,000个数据集进行了大规模分析。首先评估了当前模型与数据集文档的完整性,报告真实存在的缺陷并提出改进建议;其次分析了现有供应链结构;最后探讨了许可证现状,对比先前研究并指出该领域特有的挑战。结果表明亟需改进模型与数据集的许可证管理、增强文档支持,并实现自动化不一致性检测与验证。研究基础设施与数据集已公开,以推动后续研究。
原文摘要 · Abstract (English)
The last decade has seen widespread adoption of Machine Learning (ML) components in software systems. This has occurred in nearly every domain, from natural language processing to computer vision. These ML components range from relatively simple neural networks to complex and resource-intensive large language models. However, despite this widespread adoption, little is known about the supply chain relationships that produce these models, which can have implications for compliance and security. In this work, we conducted an extensive analysis of 760,460 models and 175,000 datasets extracted from the popular model-sharing site Hugging Face. First, we evaluate the current state of documentation in the Hugging Face supply chain, report real-world examples of shortcomings, and offer actionable suggestions for improvement. Next, we analyze the underlying structure of the existing supply chain. Finally, we explore the current licensing landscape against what was reported in previous work and discuss the unique challenges posed in this domain. Our results motivate multiple research avenues, including the need for better license management for ML models/datasets, better support for model documentation, and automated inconsistency checking and validation. We make our research infrastructure and dataset available to facilitate future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。