为1C企业系统构建首个自然语言代码检索基准与高效模型
Natural Language Code Retrieval for 1C:Enterprise: An Open Benchmark and Efficient Bi-Encoder
- 用合成数据+马特里什卡表征学习训练专用双编码器
- 在3413对真实代码查询上达0.5992的nDCG@10
- 模型轻量且隐私安全，适合工业级代码检索
自然语言代码检索是计算机科学中的快速演进任务。然而，1C:Enterprise生态系统结合了俄语语法与高度领域特定的术语，缺乏公开数据集和专用模型。我们提出一个完整的1C代码检索流水线：包含3,413个真实世界、去隐私化的查询-代码配对的开源基准、可复现的评估工具，以及一个专用双编码器。为克服标注数据稀缺问题，我们在784,057个由google/gemma-4-26B-A4B-it生成的合成三元组上进行微调，采用马特里什卡表征学习（MRL）和隐私感知分词器。因基准子集大小不同，报告平衡子集宏平均、查询加权微观平均及论坛专用结果。模型在平衡宏平均上达0.5992 nDCG@10，微观平均0.5044，论坛测试0.4617；基线架构为0.4932，google/embeddinggemma-300m为0.5404。移除所有经保守精确/13-gram重叠审计标记的样本后，平衡宏平均仍达0.6011（微观0.5010），表明训练-基准重叠不解释主要结果。将MRL截断至256维保留99.9%检索性能，同时使密集索引存储和精确相似度计算减少三倍。
原文摘要 · Abstract (English)
Natural language code retrieval is a rapidly evolving task in computer science. However, the 1C:Enterprise ecosystem combines Russian syntax with highly domain-specific terminology, for which open datasets and specialized models have been virtually non-existent. We present a comprehensive pipeline for 1C code retrieval: an open benchmark of 3,413 real-world, PII-scrubbed query-code pairs, a reproducible evaluation harness, and a specialized bi-encoder. To overcome scarce labeled data, we fine-tune on 784,057 synthetic triplets generated by google/gemma-4-26B-A4B-it from public code repositories, using Matryoshka Representation Learning (MRL) and a privacy-aware tokenizer. Because the benchmark subsets differ in size, we report balanced-subset macro, query-weighted micro, and forum-only results. Our model reaches 0.5992 balanced macro nDCG@10, 0.5044 micro, and 0.4617 on forum, versus 0.4932 macro for the baseline architecture and 0.5404 for google/embeddinggemma-300m. Removing every benchmark example flagged by the conservative exact/13-gram overlap audit leaves 0.6011 balanced macro (0.5010 micro), indicating that detected train-benchmark overlap does not explain the headline result. MRL truncation to 256 dimensions preserves 99.9% of retrieval quality while reducing dense-index storage and exact similarity arithmetic by a factor of three.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。