研究配置文件结构对代码生成器遵守指令的影响，发现结构因素影响不大，但生成函数越多越难保持一致。
Instruction Adherence in Coding Agent Configuration Files: A Factorial Study of Four File-Structure Variables
- 通过四变量因子实验，测试文件大小、指令位置等结构因素对代码生成的影响。
- 在1650次会话中未发现结构变量有显著影响，最大效应是每生成一个函数合规率降5.6%。
- 结果提示应关注任务类型和生成序列，而非配置文件结构设计。
前沿代码生成代理在会话开始时读取配置文件（CLAUDE.md、AGENTS.md、Cursor Rules），需遵循其中规范。从业者认为文件结构（如大小、指令位置、架构、相邻文件冲突）会影响其遵守程度。本研究通过系统因子实验，操控四个变量，基于1,650次Claude Code CLI会话（共16,050个函数级观测）在两个TypeScript代码库上，使用三个前沿模型（主用Sonnet 4.6，Opus 4.6作为跨模型对照，Opus 4.7因CLI版本混淆仅作描述性报告）和五个编码任务进行评估。采用混合效应模型与贝叶斯分析。四项结构变量及三组二阶交互均未在多重检验校正后显示可检测差异；文件大小与冲突的零效应获支持性贝叶斯因子（BF10介于0.05至0.10之间）；位置与架构的零效应仅为不拒绝，无贝叶斯支持。测得最大效应为会话内：每增加一个函数，合规概率下降约5.6%（OR = 0.944），但该关系非单调。该现象在第二个代码库及匹配配置的Opus 4.6上复现，且分析中首次识别，非预设假设。
原文摘要 · Abstract (English)
Frontier coding agents read configuration files (CLAUDE$.$md, AGENTS$.$md, Cursor Rules) at session start and are expected to follow the conventions inside them. Practitioners assume that structural choices (file size, instruction position, file architecture, contradictions in adjacent files) measurably affect adherence. We report a systematic factorial study of these choices using four manipulated variables, measuring compliance with a trivial target annotation across 1,650 Claude Code CLI sessions (16,050 function-level observations) on two TypeScript codebases, three frontier models (primarily Sonnet 4.6, with Opus 4.6 as a CLI-matched cross-model check and Opus 4.7 reported descriptively under a CLI-version confound), and five coding tasks. We use mixed-effects models with a Bayesian companion. None of the four structural variables or three two-way interactions produces a detectable contrast after multiple-testing correction. Size and conflict nulls are supported by affirmative-null Bayes factors (BF10 between 0.05 and 0.10); position and architecture nulls are failures to reject without Bayes-factor support. The largest effect we measured is within-session: each additional function the agent generates is associated with approximately 5.6% lower odds of compliance per step (OR = 0.944) within the session-length range we tested, though the relationship is non-monotonic rather than a constant per-step effect. This reproduces on a second TypeScript codebase and on Opus 4.6 at matched configuration; it was identified during analysis rather than pre-specified. Within the conditions tested, file-structure variables did not produce detectable contrasts; compliance varies systematically between coding tasks and across each session's sequence of generated functions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。