对比三款大模型在中文零样本关系抽取中的表现
Zero-Shot End-to-End Relation Extraction in Chinese: A Comparative Study of Gemini, LLaMA and ChatGPT
- 直接用大模型端到端识别实体与关系，无需标注数据
- ChatGPT准确率最高，Gemini推理最快，LLaMA表现最弱
- 为中文复杂任务的模型选型提供实证参考
本研究探究了多种大语言模型（LLMs）在中文零样本端到端关系抽取（RE）任务中的表现，该任务无需标注数据即可同时完成实体识别与关系抽取。尽管大模型在关系抽取中展现潜力，但以往研究多集中于英文或依赖预标注实体，其在中文关系抽取中的有效性仍不明确。为此，我们评估了ChatGPT、Gemini和LLaMA在准确率、效率和适应性方面的表现。ChatGPT整体性能最优，精度与召回率平衡良好；Gemini推理速度最快，适合实时应用；而LLaMA在准确率和延迟方面均表现较差，凸显其需进一步优化。研究结果揭示了大模型在零样本中文关系抽取中的优劣与权衡，为未来提升大模型在中文自然语言处理复杂任务中的适应性提供了基础。
原文摘要 · Abstract (English)
This study investigates the performance of various large language models (LLMs) on zero-shot end-to-end relation extraction (RE) in Chinese, a task that integrates entity recognition and relation extraction without requiring annotated data. While LLMs show promise for RE, most prior work focuses on English or assumes pre-annotated entities, leaving their effectiveness in Chinese RE largely unexplored. To bridge this gap, we evaluate ChatGPT, Gemini, and LLaMA based on accuracy, efficiency, and adaptability. ChatGPT demonstrates the highest overall performance, balancing precision and recall, while Gemini achieves the fastest inference speed, making it suitable for real-time applications. LLaMA underperforms in both accuracy and latency, highlighting the need for further adaptation. Our findings provide insights into the strengths and limitations of LLMs for zero-shot Chinese RE, shedding light on trade-offs between accuracy and efficiency. This study serves as a foundation for future research aimed at improving LLM adaptability to complex linguistic tasks in Chinese NLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。