用自然语言自动生成操作系统安全策略代码，准确率超98%。
Text2DSL: LLM-Based Code Generation for Domain-Specific Languages
- 通过形式化语法和接口规范注入提示词，提升生成代码准确性。
- 在4204个样本上，语法正确率达98.6%-99.4%，结构正确率提升35.5个百分点。
- 适用于无须微调模型的领域专用语言生成，适合安全策略开发者使用。
领域专用语言（DSL）广泛用于管理系统安全策略，但手动编写规则需高技能且易出错。本文将自然语言到DSL代码的自动转换定义为独立任务，提出PolkitBench数据集，包含4,204对经三阶段AST验证的自然语言-Polkit规则对。在两个MoE模型（GigaChat-10B-A1.8B与Nemotron-3-Nano-30B-A3B）上进行受控提示实验，结果显示：引入上下文（BNF语法、API说明、允许标识符列表）后，语法正确率提升至98.6%-99.4%，结构正确率提高9.7至35.5个百分点，CodeBLEU得分提升60%至95%。该效果在不同规模和来源的模型间具一致性，表明对于Text2DSL任务，仅通过提示词注入目标语言规范即可显著提升生成质量，无需模型微调。
原文摘要 · Abstract (English)
Domain-specific languages (DSLs) are widely used for managing operating system security policies, yet manually authoring rules in such languages demands high expertise and is error-prone. This paper formalises the task of automatic DSL code generation from natural language descriptions - Text2DSL - as a distinct problem class, separate from Text-to-SQL and general-purpose code generation. We introduce the PolkitBench dataset comprising 4,204 verified natural-language-to-Polkit-rule pairs, each validated through a three-level AST-based pipeline. Controlled prompt experiments on two MoE models of different scale and provenance - GigaChat-10B-A1.8B (1.8B active parameters) and Nemotron-3-Nano-30B-A3B (3B active) - demonstrate the critical role of structured context (BNF grammar, API specification, permitted identifier vocabulary) for LLM-based DSL code generation. Across both models, supplying context raises syntactic validity to 98.6-99.4%, structural validity by +9.7 to +35.5 pp, and the CodeBLEU score by +60% to +95%. The consistency of the effect across models of different scale and provenance indicates that, for the Text2DSL class of problems, injecting a formal target-language specification into the prompt context is a robust enabling factor for high-quality generation without model fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。