用贝叶斯优化融合文档与知识图谱，提升领域文本表示效率和可解释性。
FuDoBa: Fusing Document and Knowledge Graph-based Representations with Bayesian Optimisation
- 通过贝叶斯优化融合LLM嵌入与领域知识图谱
- 在6个数据集上达到或超越纯LLM基线性能
- 生成低维、可解释的融合权重，适合领域应用
基于大语言模型（LLM）的文档嵌入在文档表示任务中表现优异，但其高维、计算成本高的特性常导致泛化过度或效率低下，难以适应特定领域需求。为此，本文提出FuDoBa方法，利用贝叶斯优化将LLM嵌入与本地及外部知识库（如WikiData）中的结构化领域知识进行融合，生成低维、任务相关性强的表示，降低训练复杂度，并获得可解释的早期融合权重，从而提升分类性能。我们在两个领域的六个数据集上验证了该方法的有效性，结果表明：当搭配稳健的AutoML分类器时，该表示学习方法在性能上可媲美甚至超越仅使用专有LLM嵌入的基线。
原文摘要 · Abstract (English)
Building on the success of Large Language Models (LLMs), LLM-based representations have dominated the document representation landscape, achieving great performance on the document embedding benchmarks. However, the high-dimensional, computationally expensive embeddings from LLMs tend to be either too generic or inefficient for domain-specific applications. To address these limitations, we introduce FuDoBa a Bayesian optimisation-based method that integrates LLM-based embeddings with domain-specific structured knowledge, sourced both locally and from external repositories like WikiData. This fusion produces low-dimensional, task-relevant representations while reducing training complexity and yielding interpretable early-fusion weights for enhanced classification performance. We demonstrate the effectiveness of our approach on six datasets in two domains, showing that when paired with robust AutoML-based classifiers, our proposed representation learning approach performs on par with, or surpasses, those produced solely by the proprietary LLM-based embedding baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。