对比BERT与GPT中不同池化方法的性能，为情感分析提供选型参考。
Comparative Analysis of Pooling Mechanisms in LLMs: A Sentiment Analysis Perspective
- 在BERT和GPT上测试均值、最大值、加权求和三种池化方法
- 不同模型对池化方式敏感度不同，影响情感分析准确率
- 研究结果可指导下游任务中池化策略的合理选择
大型语言模型（LLM）通过Transformer架构在自然语言处理任务中实现顶尖性能。以BERT和GPT为代表的模型依赖池化层将词元级嵌入聚合为句子级表示。常见的池化机制如均值、最大值和加权求和在该过程中起关键作用。尽管广泛使用，这些方法在不同LLM架构上的比较性能仍不明确。本文针对句级情感分析任务，系统评估了均值、最大值和加权求和在BERT与GPT两类模型上的表现。实验表明，各池化方法的优劣因模型架构和任务需求而异。研究强调应根据具体应用需求选择池化策略，挑战了对池化操作的普遍假设。结果为优化基于LLM的下游任务提供了可操作的洞见。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have revolutionized natural language processing (NLP) by delivering state-of-the-art performance across a variety of tasks. Among these, Transformer-based models like BERT and GPT rely on pooling layers to aggregate token-level embeddings into sentence-level representations. Common pooling mechanisms such as Mean, Max, and Weighted Sum play a pivotal role in this aggregation process. Despite their widespread use, the comparative performance of these strategies on different LLM architectures remains underexplored. To address this gap, this paper investigates the effects of these pooling mechanisms on two prominent LLM families -- BERT and GPT, in the context of sentence-level sentiment analysis. Comprehensive experiments reveal that each pooling mechanism exhibits unique strengths and weaknesses depending on the task's specific requirements. Our findings underline the importance of selecting pooling methods tailored to the demands of particular applications, prompting a re-evaluation of common assumptions regarding pooling operations. By offering actionable insights, this study contributes to the optimization of LLM-based models for downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。