用开发者看代码时的视线数据,让大模型更懂代码重点。
EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarization
- 引入人类注视模式作为注意力先验,动态调整代码词元权重。
- 在多个模型上提升效果,最高达13.17%的BLEU-4增益。
- 无需微调,轻量插入即可增强模型对代码语义的关注能力。
代码摘要任务旨在生成源代码的自然语言描述,对软件理解和维护至关重要。尽管大语言模型(LLMs)在此任务上已取得显著进展,但一个开放问题仍存:人类在代码理解中的专业知识能否进一步指导并增强这些模型?我们提出EyeLayer,一种轻量级注意力增强模块,将人类眼动模式(作为人类专业性的代理)融入基于LLM的代码摘要中。EyeLayer通过多模态高斯混合模型建模开发人员读代码时的注意力分布,依据学习得到的参数(μ_i, σ_i²)重新分配词元嵌入,捕捉开发者关注的位置与强度。该设计可从眼动数据中学习可泛化的注意力先验,并无缝集成至LLM中,不破坏原有表示。我们在多种模型家族(如LLaMA-3.2、Qwen3和CodeBERT)上评估EyeLayer,覆盖不同规模与架构。结果表明,EyeLayer在标准指标上持续优于强基线微调方法,最大提升达13.17%的BLEU-4。这证明人类注视模式蕴含互补的注意力信号,能有效提升LLM的语义聚焦能力,并在多样模型间良好迁移,适用于代码摘要任务。
原文摘要 · Abstract (English)
Code summarization is the task of generating natural language descriptions of source code, which is critical for software comprehension and maintenance. While large language models (LLMs) have achieved remarkable progress on this task, an open question remains: can human expertise in code understanding further guide and enhance these models? We propose EyeLayer, a lightweight attention-augmentation module that incorporates human eye-gaze patterns, as a proxy of human expertise, into LLM-based code summarization. EyeLayer models human attention during code reading via a Multimodal Gaussian Mixture, redistributing token embeddings based on learned parameters (μ_i, σ_i^2) that capture where and how intensively developers focus. This design enables learning generalizable attention priors from eye-tracking data and incorporating them into LLMs seamlessly, without disturbing existing representations. We evaluate EyeLayer across diverse model families (i.e., LLaMA-3.2, Qwen3, and CodeBERT) covering different scales and architectures. EyeLayer consistently outperforms strong fine-tuning baselines across standard metrics, achieving gains of up to 13.17% on BLEU-4. These results demonstrate that human gaze patterns encode complementary attention signals that enhance the semantic focus of LLMs and transfer effectively across diverse models for code summarization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。