用自回归模型生成代码嵌入，实现跨语言代码检索与问答
Efficient Code Embeddings from Code Generation Models
- 基于文本代码联合预训练的自回归模型，通过末尾标记池化生成嵌入
- 在小型模型规模下达到顶尖性能，支持自然语言查代码和跨语言相似性匹配
- 适合需要高效代码搜索与理解的开发者工具、智能编程助手
jina-code-embeddings 是一套新型代码嵌入模型，旨在通过自然语言查询检索代码、回答技术问题，并识别跨编程语言的语义相似代码片段。该模型采用在文本和代码上联合预训练的自回归骨干网络，通过末尾标记池化生成嵌入表示。文章详细说明了训练方法，并证明了即使模型规模相对较小，仍可实现当前最优性能，验证了该构建方式的有效性。
原文摘要 · Abstract (English)
jina-code-embeddings is a novel code embedding model suite designed to retrieve code from natural language queries, perform technical question-answering, and identify semantically similar code snippets across programming languages. It makes innovative use of an autoregressive backbone pre-trained on both text and code, generating embeddings via last-token pooling. We outline the training recipe and demonstrate state-of-the-art performance despite the relatively small size of the models, validating this approach to code embedding model construction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。