用大模型自动整理和生成标准化模型卡片,提升透明度与可比性。
Automatic Model Card Generation Using an LLM

- 基于大模型重构现有卡片为统一模板,确保结构清晰
- 生成卡片语义相似度达0.9,超半数内容完全正确
- 依赖论文等资源质量,适合需要快速生成文档的团队
模型卡片是总结机器学习模型关键信息的结构化文档,有助于提升透明度、可用性和问责性。然而,现有卡片常缺乏一致结构,许多模型甚至无卡片,导致比较与理解困难。本文提出两项贡献:首先,MCTidy 是一种基于大模型的方法,将现有模型卡片重排至标准模板,以增强清晰度与可比性;其次,MCGenie 是一个基于大模型的系统,可直接从模型仓库数据生成模型卡片。我们对48个Hugging Face模型卡片应用MCTidy,评估信息保留、段落对齐、幻觉率和运行稳定性,结果表明信息保留率高,文本损失小,段落分配准确,幻觉罕见且多出现在描述性部分,跨次运行稳定。通过MCGenie为相同48个模型生成卡片,评估语义相似性、事实正确性和输入资源敏感性,生成卡片平均语义相似度约0.9,超过一半完全正确,其余错误多为轻微问题。生成质量显著依赖支持性资源,尤其是相关论文。整体表明,大模型方法具备实现规模化、标准化模型文档的潜力。
原文摘要 · Abstract (English)
Model cards are structured documents that summarize key information about machine learning models to improve transparency, usability, and accountability. However, they often lack a consistent structure, and many models provide no model cards, making comparison and interpretation difficult. This paper presents two contributions. First, we propose MCTidy, an LLM-based approach that reorganizes existing model cards into a standardized template to improve clarity and comparability. Second, we introduce MCGenie, an LLM-based system that generates model cards directly from model repository data. We apply MCTidy to 48 Hugging Face model cards and evaluate information retention, section alignment, hallucination, and stability. Our findings show high information retention with minimal textual loss, accurate section assignment, rare hallucinations primarily in descriptive sections, and strong stability across runs. We assess MCGenie by generating model cards for the same 48 models and assessing semantic similarity, factual correctness, and sensitivity to input resources. The generated model cards achieved high semantic similarity (mean around 0.9); over half were fully correct, and most remaining errors were minor. Generation quality depended strongly on the availability of supporting resources, particularly associated papers. Overall, our findings demonstrate the potential of LLM-based methods to enable scalable, standardized model card documentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。