arXiv:2503.10658cs.CLcs.LG2025-03中稿 · JCDL 2024被引 37

用大模型自动提炼科研论文局限性,生成可读标题与摘要。

LimTopic: LLM-based Topic Modeling and Text Summarization for Analyzing Scientific Articles limitations

  • 结合BERTopic与大模型生成局限性主题标题和句子。
  • GPT-4在主题建模与摘要生成上表现最优,轮廓系数与连贯性更高。
  • 适合研究者、审稿人快速理解论文不足,指导后续研究。

科研论文的局限性部分对揭示研究边界与不足至关重要,有助于引导未来研究与改进方法。本文提出LimTopic,利用大语言模型(LLMs)对科学论文中的局限性内容进行主题建模与文本摘要。该方法通过整合BERTopic与LLM,为每个主题生成标题和主题句,并进一步利用LLM生成简洁、通用的主题摘要。实验涵盖提示工程、LLM微调及与BERTopic的集成,测试了多种LLM在主题建模与摘要任务中的表现。结果表明,BERTopic与GPT-4的组合在主题建模中取得最佳轮廓系数与连贯性得分,且GPT-4在文本摘要任务中优于其他模型。

原文摘要 · Abstract (English)

The limitations sections of scientific articles play a crucial role in highlighting the boundaries and shortcomings of research, thereby guiding future studies and improving research methods. Analyzing these limitations benefits researchers, reviewers, funding agencies, and the broader academic community. We introduce LimTopic, a strategy where Topic generation in Limitation sections in scientific articles with Large Language Models (LLMs). Here, each topic contains the title and Topic Summary. This study focuses on effectively extracting and understanding these limitations through topic modeling and text summarization, utilizing the capabilities of LLMs. We extracted limitations from research articles and applied an LLM-based topic modeling integrated with the BERtopic approach to generate a title for each topic and Topic Sentences. To enhance comprehension and accessibility, we employed LLM-based text summarization to create concise and generalizable summaries for each topic Topic Sentences and produce a Topic Summary. Our experimentation involved prompt engineering, fine-tuning LLM and BERTopic, and integrating BERTopic with LLM to generate topics, titles, and a topic summary. We also experimented with various LLMs with BERTopic for topic modeling and various LLMs for text summarization tasks. Our results showed that the combination of BERTopic and GPT 4 performed the best in terms of silhouette and coherence scores in topic modeling, and the GPT4 summary outperformed other LLM tasks as a text summarizer.

主题建模大模型论文分析摘要生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。