构建首个可落地的开源技能库,提升大模型代理实际任务表现。
SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents

- 通过多阶段筛选整合82万条技能,保留9.6万条高质量可用技能。
- 在三个基准上平均提升7.5个百分点,最大增益来自技能覆盖与调用机制优化。
- 适合研究技能聚合、评估及部署大模型代理的开发者和研究人员。
Agent技能(SKILL文件)是封装可复用过程知识以扩展大模型代理能力的流行方式。尽管公共仓库中积累了大量技能,但这些资源分散、重复且质量参差,其实际价值尚不明确。本文提出SkillCorpus框架,实现对开放技能生态的规模化聚合、整理、匹配与评估。通过多阶段流程从约82.1万条爬取技能中筛选出96,401条,按16类分类体系与实用、鲁棒性、安全性三维度标注,并配套微调的检索-选择模块实现任务匹配。在SkillsBench、GDPVal、QwenClawBench三个基准,两个工具链,两个开源后端上进行端到端评估,包含前沿鲁棒性测试。集成SkillCorpus后在所有基准上均获得稳定提升,最大增幅达7.5个百分点。操作分析表明收益源于技能覆盖率与工具链适配边界。该工作是首个系统性回答‘有组织的社区技能库何时能提升真实代理任务’的实证研究。数据集、模型与代码已公开于https://github.com/EverMind-AI/SkillCorpus。
原文摘要 · Abstract (English)
Agent skills, SKILL files that package reusable procedural knowledge for an LLM agent, are a popular mechanism for extending agent capabilities. Public repositories now host them in large and growing numbers, yet these artifacts are fragmented, redundant, and uneven in quality, and their value in practice is unclear. A core question remains open, namely how to consolidate this open-source SKILL ecosystem into a single usable corpus, and what bounds its benefit on real-world agent tasks. We present SkillCorpus, a framework that aggregates, curates, matches, and evaluates the open skill ecosystem at scale. It filters ~821,000 crawled skills through a multi-stage pipeline into 96,401 skills organised by a 16-class taxonomy and three quality facets (utility, robustness, safety), and pairs them with a fine-tuned retrieval-and-selection stack that matches task-relevant skills. We evaluate end-to-end across three benchmarks (SkillsBench, GDPVal, QwenClawBench), two harnesses, and two open backbones with a frontier robustness check. Integrating SkillCorpus yields consistent gains across all three benchmarks, largest on SkillsBench (+7.5 pp). An operational analysis traces the gains to a coverage boundary and a harness boundary. SkillCorpus is, to our knowledge, the first end-to-end account of when a curated, retrieval-served community corpus improves real agent tasks, and where it does not. The dataset, models, and code are available at https://github.com/EverMind-AI/SkillCorpus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。