用程序员眼动数据提升代码大模型训练效果
Enhancing Code LLM Training with Programmer Attention
- 通过眼动路径增强扩充注意力数据集
- 提炼出可学习的注意力模式，提升模型理解力
- 融合人类注意力信号，显著优化代码生成性能
人类注意力为代码大模型训练提供了宝贵但未被充分利用的信号，超越了纯机器驱动的注意力机制。尽管眼动数据收集复杂且成本高，但将其系统性地用于代码大模型训练仍进展有限。为此，我们提出一个完整的流水线，涵盖数据增强与基于奖励的微调。具体包括：(1) 提出一种眼动路径增强方法，扩大程序员注意力数据集；(2) 设计模式抽象步骤，将原始注视点转化为可学习的注意力模式；(3) 采用基于奖励的策略，将这些洞察直接整合进CodeT5的监督微调过程。实验结果显示，在CodeXGlue代码摘要任务上，CodeBLEU指标提升7.16分，证明融合人类与机器注意力能显著增强代码智能。我们希望本工作能推动下一代AI for Software Engineering中以人为本方法的探索。
原文摘要 · Abstract (English)
Human attention provides valuable yet underexploited signals for code LLM training, offering a perspective beyond purely machine-driven attention. Despite the complexity and cost of collecting eye-tracking data, there has also been limited progress in systematically using these signals for code LLM training. To address both issues, we propose a cohesive pipeline spanning augmentation and reward-based fine-tuning. Specifically, we introduce (1) an eye-tracking path augmentation method to expand programmer attention datasets, (2) a pattern abstraction step that refines raw fixations into learnable attention motifs, and (3) a reward-guided strategy for integrating these insights directly into a CodeT5 supervised fine-tuning process. Our experiments yield +7.16 in CodeBLEU on the CodeXGlue benchmark for code summarization, underscoring how uniting human and machine attention can boost code intelligence. We hope this work encourages broader exploration of human-centric methods in next-generation AI4SE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。