CodeWatcher 轻量采集开发者使用大模型编程时的实时交互数据。
CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs
- 通过 VS Code 插件无感记录代码插入、删除、复制等操作
- 支持会话级行为重建，可分析大模型在编码中的实际使用时机
- 适合研究开发者效率、人机协作与负责任 AI 的团队
理解开发者如何使用代码生成工具（CGTs）需要细粒度的实时编程行为数据，但传统采集方式常干扰工作流。我们提出 CodeWatcher，一个轻量级、无侵入的客户端-服务器系统，可从 Visual Studio Code 编辑器中捕获细粒度交互事件。该系统记录由 CGTs 引起的插入、删除、复制粘贴及焦点切换等语义化事件，实现对开发活动的持续监控而不改变用户工作流程。系统包含 VS Code 插件、基于 Python 的 RESTful API 和 MongoDB 后端，均容器化部署，具备可扩展性。通过结构化和时间戳标记每个事件，CodeWatcher 支持事后重构编码会话，推动对开发者行为的深入分析，包括大模型在开发过程中的使用模式。该基础设施对负责任 AI 研究、开发者生产力评估和以人为核心的 CGT 评价具有重要意义。
原文摘要 · Abstract (English)
Understanding how developers interact with code generation tools (CGTs) requires detailed, real-time data on programming behavior which is often difficult to collect without disrupting workflow. We present \textit{CodeWatcher}, a lightweight, unobtrusive client-server system designed to capture fine-grained interaction events from within the Visual Studio Code (VS Code) editor. \textit{CodeWatcher} logs semantically meaningful events such as insertions made by CGTs, deletions, copy-paste actions, and focus shifts, enabling continuous monitoring of developer activity without modifying user workflows. The system comprises a VS Code plugin, a Python-based RESTful API, and a MongoDB backend, all containerized for scalability and ease of deployment. By structuring and timestamping each event, \textit{CodeWatcher} enables post-hoc reconstruction of coding sessions and facilitates rich behavioral analyses, including how and when CGTs are used during development. This infrastructure is crucial for supporting research on responsible AI, developer productivity, and the human-centered evaluation of CGTs. Please find the demo, diagrams, and tool here: https://osf.io/j2kru/overview.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。