Mellum在IDE中实现高效多文件代码补全,兼顾质量与实时性。
Mellum: Production-Grade in-IDE Contextual Code Completion with Multi-File Project Understanding
- 基于多阶段训练与数据精炼,提升代码补全准确率
- 支持项目级上下文,补全结果更符合实际开发场景
- 轻量模型满足生产环境低延迟需求,适合开发者直接使用
我们提出Mellum模型系列,是专为JetBrains IDE设计的开放权重代码补全模型。模型拥有40亿参数,采用Llama风格架构,基于约4万亿标记的开源多语言代码进行预训练。研究显示:(i) 精心的数据筛选与分阶段训练显著提升模型质量;(ii) 编辑器关键能力如上下文打包对高质量建议至关重要;(iii) 紧凑、任务聚焦的模型可满足交互式补全的成本与延迟要求。本文介绍从数据治理到部署的完整工业级流水线:规范的数据管理、包含填空和项目上下文的监督微调,以及基于真实场景反馈的直接偏好优化。评估涵盖大规模离线基准与生产环境在线遥测数据。Mellum已在HuggingFace以Apache-2.0许可证发布,附带可复现的公开模型卡,为研究人员提供落地生产实践蓝图。
原文摘要 · Abstract (English)
We present the Mellum models family, open-weight code completion models designed for interactive use in JetBrains IDEs. Mellums have 4B parameters, adopt a Llama-style architecture, and are pre-trained on ~4T tokens of permissively licensed, multi-language code. Our studies show that (i) careful data curation and staged training significantly improve the model's quality, (ii) editor-critical capabilities such as context packing are necessary for high-quality suggestions, and (iii) a compact, task-focused model can meet the cost and latency constraints of interactive completion. In the paper, we describe an end-to-end industrial pipeline for producing contextualized in-editor completion: disciplined data governance, multi-stage training that includes fill-in-the-middle and project context via supervised fine-tuning, and alignment via direct preference optimization using feedback from real-world scenarios. Our quality evaluations include both large-scale offline benchmarks and online telemetry from production deployments in JetBrains IDEs. Mellums are released under the Apache-2.0 license on HuggingFace, with a public model card providing a reproducible reference for practitioners. Our experience offers a pragmatic blueprint for taking a focused, open model from a research prototype to at scale production for hundreds of thousands of users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。