用现有大模型嵌入实现高效少样本表格分类,无需重新训练。
Language Model Representations for Efficient Few-Shot Tabular Classification
- 直接使用大模型对表格行生成语义嵌入,构建轻量级分类框架。
- 通过去除共性特征和温度校准,使嵌入表示性能提升至领先水平。
- 适合资源有限但需快速部署表格分类的场景,如电商、知识库应用。
网络中存在大量结构化数据,如产品目录、知识库和科学数据集,但其结构与语义差异大,难以统一处理。大型语言模型(LLMs)已广泛用于语义搜索等任务,这引发一个问题:能否利用已部署的LLMs对网页原生表格(如产品目录、知识库导出、科学数据门户)进行分类,避免专用模型或大规模重训练?本文提出轻量级框架TaRL,直接使用表中每行的语义嵌入进行少样本分类。我们发现,简单使用嵌入表现不佳;通过移除所有嵌入的公共成分并校准softmax温度,可显著释放潜力。一个基于手工特征的元学习器可预测最优温度。该方法在低数据场景(k ≤ 32)下达到与当前最优模型相当的性能,验证了复用现有LLM基础设施实现高效语义驱动表格理解的可行性。
原文摘要 · Abstract (English)
The Web is a rich source of structured data in the form of tables, from product catalogs and knowledge bases to scientific datasets. However, the heterogeneity of the structure and semantics of these tables makes it challenging to build a unified method that can effectively leverage the information they contain. Meanwhile, Large language models (LLMs) are becoming an increasingly integral component of web infrastructure for tasks like semantic search. This raises a crucial question: can we leverage these already-deployed LLMs to classify structured data in web-native tables (e.g., product catalogs, knowledge base exports, scientific data portals), avoiding the need for specialized models or extensive retraining? This work investigates a lightweight paradigm, $\textbf{Ta}$ble $\textbf{R}$epresentation with $\textbf{L}$anguage Model~($\textbf{TaRL}$), for few-shot tabular classification that directly utilizes semantic embeddings of individual table rows. We first show that naive application of these embeddings underperforms compared to specialized tabular models. We then demonstrate that their potentials can be unlocked with two key techniques: removing the common component from all embeddings and calibrating the softmax temperature. We show that a simple meta-learner, trained on handcrafted features, can learn to predict an appropriate temperature. This approach achieves performance comparable to state-of-the-art models in low-data regimes ($k \leq 32$) of semantically-rich tables. Our findings demonstrate the viability of reusing existing LLM infrastructure for efficient semantics-driven pathway to reuse existing LLM infrastructure for Web table understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。