arXiv:2503.15117cs.CL2025-03AAAI被引 11

通过精准编辑关键神经元,高效适配大模型做情感分析。

Exploring Model Editing for LLM-based Aspect-Based Sentiment Classification

  • 识别中层神经元状态对情感极性判断的关键作用
  • 仅更新少量参数即达主流方法性能,参数量减少显著
  • 适合需要高效、可解释微调的NLP应用场景

模型编辑旨在以可解释策略选择性地更新神经网络的少量参数,从而显著降低大语言模型(LLMs)的适应成本。本文研究利用模型编辑实现大模型在基于方面的情感分类任务中的高效适配。通过因果干预,我们追踪并确定了影响模型预测的关键神经元隐藏状态。通过对LLM各组件进行干预与恢复操作,识别出这些组件在基于方面情感分类中的重要性。研究发现,特定的一组中层表示对检测给定方面词的情感极性至关重要。基于此,我们提出一种聚焦于这些关键部分的模型编辑方法,实现了更高效的模型适配。领域内与跨领域实验表明,该方法在显著减少可训练参数的前提下,达到与当前最强方法相媲美的性能,凸显了一种更高效且可解释的微调策略。

原文摘要 · Abstract (English)

Model editing aims at selectively updating a small subset of a neural model's parameters with an interpretable strategy to achieve desired modifications. It can significantly reduce computational costs to adapt to large language models (LLMs). Given its ability to precisely target critical components within LLMs, model editing shows great potential for efficient fine-tuning applications. In this work, we investigate model editing to serve an efficient method for adapting LLMs to solve aspect-based sentiment classification. Through causal interventions, we trace and determine which neuron hidden states are essential for the prediction of the model. By performing interventions and restorations on each component of an LLM, we identify the importance of these components for aspect-based sentiment classification. Our findings reveal that a distinct set of mid-layer representations is essential for detecting the sentiment polarity of given aspect words. Leveraging these insights, we develop a model editing approach that focuses exclusively on these critical parts of the LLM, leading to a more efficient method for adapting LLMs. Our in-domain and out-of-domain experiments demonstrate that this approach achieves competitive results compared to the currently strongest methods with significantly fewer trainable parameters, highlighting a more efficient and interpretable fine-tuning strategy.

模型编辑情感分析大模型微调可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。