让表格模型理解特征语义，提升小数据场景下的预测准确率。
Enhancing Tabular Learners with Context-Aware Semantic Embeddings
- 用上下文化嵌入技术，让模型理解特征名和单元格的语义。
- 在低数据量下性能显著提升，尤其在CARTE等基准上超越基线。
- 适合需要理解复杂语义的表格数据任务，如医疗、金融建模。
现代表格学习模型擅长捕捉统计模式，但常在语义空白中运行，将文本特征视为离散符号，忽视特征名或单元格内容中的丰富语义。我们提出CASE（上下文感知语义嵌入）框架，弥合大型语言模型（LLMs）的语义理解与表格学习者的统计能力之间的差距。不同于现有方法孤立嵌入每行数据，CASE采用上下文化策略：预先用代表性行填充基于Gemma 3的定制表格语言模型的键值缓存，建立数据集语义的持续锚点。这确保生成的行嵌入动态上下文相关，解决语义歧义，并将表示锚定在领域特定上下文中。我们在CARTE、TextTab和TabArena等多个基准上的实验表明，CASE显著提升了表格学习者在语义丰富数据集上的表现，尤其是在低数据环境下。
原文摘要 · Abstract (English)
While modern tabular learners excel at capturing statistical patterns, they frequently operate in a semantic vacuum, treating textual features as discrete symbols, ignoring the rich semantics inherent in feature names or cell entries. We propose CASE (Context-Aware Semantic Embeddings), a novel framework that bridges the gap between the semantic understanding of Large Language Models (LLMs) and the statistical capabilities of tabular learners. Unlike existing methods that embed rows in isolation, CASE utilizes a contextualization strategy: we pre-fill the KV cache of a custom-trained Gemma 3-based Tabular Language Model with a representative sample of rows to establish a persistent anchor of the dataset's semantics. This ensures that generated row embeddings are dynamically contextualized, resolving semantic ambiguities and anchoring representations in domain-specific context. Our experiments across several benchmarks (CARTE, TextTab, and TabArena) demonstrate that CASE substantially improves the performance of tabular learners on semantically rich datasets, particularly in low-data regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。