研究AI编程工具中的规则如何演变,发现改规则能显著提升代码合规性。
Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study

- 通过挖掘83个开源项目,提炼出5大类25小类规则分类体系。
- 规则更新后代码合规率从49.14%升至72.13%,平均提升22.99%。
- 开发者实际用规则多为格式规范,但更希望控制架构约束。
AI编程工具引入了“规则”这一新型软件资产,使开发者可将项目特定约束持久注入大语言模型上下文。尽管规则对对齐AI行为与开发意图至关重要,其分类、演化及实际影响仍不清晰。本研究通过混合方法对AI IDE规则开展实证分析:从83个开源项目中提取7,310条规则,构建涵盖5个主类与25个子类的完整分类体系;结合99名从业者的调查反馈进行三角验证。结果显示,开发者虽重视架构约束,但仓库中多数规则仍为低层级流程与格式化规则。对1,540次规则演化事件分析发现,规则频繁更新,主要由构造性上下文扩展(29.17%)和丰富(26.59%)驱动;而开发者自述修改规则多为修正AI错误(77.78%),倾向新增负向约束而非修改旧规则。对160次演化事件的合规性评估表明,规则更新使软件产物合规率平均提升22.99%(从49.14%增至72.13%)。研究结果为优化提示策略及设计自动化冲突检测与上下文管理机制提供实证支持。
原文摘要 · Abstract (English)
The adoption of AI-powered Integrated Development Environments (AI IDEs) has introduced "Rules" as a novel software artifact, allowing developers to persistently inject project-specific constraints and architectural guidelines into the context of Large Language Models (LLMs). Despite their role in aligning AI behavior with developer intent, the taxonomy, evolution, and practical impact of these rules remain largely unexplored. To bridge this gap, we conducted a mixed-methods empirical study on AI IDE rules. By mining 83 open-source projects and extracting 7,310 rules, we established a comprehensive taxonomy comprising 5 primary and 25 secondary categories. We then triangulated these artifacts with survey responses from 99 practitioners. Our analysis identified a contrast between developer priorities and actual configurations: while practitioners rate architectural constraints as highly important, rule files in repositories primarily consist of low-level workflow and code formatting constraints. Furthermore, our analysis of 1,540 rule evolution events revealed that rules are updated frequently. Repository data further indicate that rule evolution is primarily driven by constructive context expansions (29.17%) and enrichments (26.59%). In contrast, surveyed developers reported modifying rules primarily to correct AI errors (77.78%), typically by adding new negative constraints rather than editing existing ones. Finally, an artifact compliance assessment of 160 rule evolution events revealed that updating rules significantly improves the adherence of software artifacts, with the average artifact compliance rate increasing by 22.99% (from 49.14% to 72.13%) following an update. Our study provides empirical insights that can help developers optimize prompting strategies and guide tool builders in designing automated conflict-detection and context-management mechanisms for AI IDEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。