用开发者交互历史生成个性化技能,发现通用技能更有效。
Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories

- 从交互记录中提取开发者偏好,生成可复用的个性化技能
- 通用技能比个性化技能带来更大且更稳定的性能提升
- 仅当开发者偏好频繁出现时,个性化技能才真正有效
基于大语言模型的编码代理已从代码补全工具演变为解决复杂软件工程任务的助手。随着开发者与代理持续互动,其偏好逐渐显现,可通过重用这些偏好来优化代理行为,减少重复修正,提升协作效率。代理技能是一种无需修改模型参数即可传递经验的轻量机制。然而,现有研究多聚焦于任务特定技能,尚不清楚从交互历史中提炼的开发者特定技能是否能泛化到未来任务。本文提出一种从交互轨迹中提取可复用开发者偏好的框架:首先通过规则引导的自举和证据支撑的精炼生成个性化技能,再利用基于交互式、轨迹条件的LLM模拟人类开发者的可复现回放框架进行评估。我们在13名开发者的206个真实交互会话上开展实验,对比个性化技能与无技能、通用技能及他人技能基线。结果显示,个性化技能对无技能基线仅有微小且不一致的改进,而跨开发者聚合的通用技能则带来最大且最一致的提升。进一步分析表明,当开发者偏好在历史中频繁出现,尤其是存在多个与未来任务相关的示例时,个性化技能效果更优。该研究提供了开发者个性化有效性的实证依据,表明广泛可迁移的过程知识比特定开发者偏好信号更具鲁棒性。
原文摘要 · Abstract (English)
Large language model (LLM)-powered agents have rapidly evolved from code-completion tools into solvers of complex software engineering tasks. As developers collaborate with coding agents over time, their preferences emerge through repeated interactions and can be used to adapt agent behavior to better meet individual developers' needs. Capturing and reusing these preferences may reduce repeated corrections and improve developer-agent collaboration. Agent skills provide a lightweight mechanism for transferring experience without modifying model parameters. However, existing work primarily focuses on task-specific skills, and it remains unclear whether developer-specific skills distilled from interaction histories can generalize to future tasks. We propose a framework for extracting reusable developer preferences from interaction traces. It first generates personalized skills through rule-based bootstrapping and evidence-grounded refinement, and then evaluates them using a reproducible replay framework with an interactive, trajectory-conditioned LLM-based human developer simulator. We conduct an experiment on 206 real-world developer-agent sessions from 13 developers and compare personalized skills against no-skill, generic-skill, and other-user-skill baselines. Personalized skills provide small and inconsistent improvements over the no-skill baseline, whereas generic skills pooled across developers achieve the largest and most consistent gains. Further analysis suggests that personalized skills become more effective when developer preferences appear frequently, particularly when their histories contain multiple examples relevant to future tasks. These findings provide empirical insights into when developer-specific personalization is effective and demonstrate that broadly transferable procedural knowledge can be more robust than developer-specific preference signals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。