22个专利嵌入模型在多任务测试中发现：不同任务需不同微调策略，单一领域训练难跨域使用。
Benchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering
- 按任务定制微调方法：检索用对齐策略，分类与聚类用联合信号策略
- 跨领域检索性能下降55%-65%，强模型在零样本下更易受损失
- 模型规模扩展不一致，同家族有效，跨家族表现随机
针对专利嵌入技术的实际应用，提出两个核心问题：(i) 是否存在一种通用微调方案适用于所有下游任务？(ii) 在一个专利领域上微调是否足以支持其他领域的应用？通过对22个预训练嵌入模型（参数量从2200万到120亿）在信息检索、分类和聚类三项任务上的评估，使用113,148份世界知识产权组织（WIPO）辅助技术专利数据（含46,069条引用查询）及外部DAPFAM数据集，发现两项结果挑战了现有认知。首先，最优微调策略依赖于任务：跨截面对齐（配方R3）使检索性能提升7.1% nDCG@10；联合信号（配方R4）则分别在分类（+7.1 F1）和聚类（+10.9 V-measure）上表现最佳；匹配数据控制实验排除了训练数据量差异的影响。其次，单领域微调显著削弱跨领域检索能力：在DAPFAM语料中，8种组合中有5种因单领域训练导致跨域性能下降，尤其强零样本模型受损最重。同家族模型规模扩展稳定（如Qwen3 0.6B→4B→8B；Llama-Nemotron 1B→8B），但跨家族扩展混乱；120亿参数的KaLM-Gemma3在TAC检索中仅排名第8，优于前缀修改。标题+摘要+权利要求是最优文本组合，所有模型在域外性能上均存在55%-65%差距，且无法通过混合BM25-密集融合缓解。代码与评估框架已公开。
原文摘要 · Abstract (English)
Two questions regarding practitioners' use of patent embeddings arise: (i) Does one fine-tuning recipe suffice for all downstream applications? (ii) Is fine-tuning on one patent landscape sufficient for downstream application on other landscapes? By evaluating 22 pre-trained embedding models (ranging from 22M to 12B parameters) on three tasks -- information retrieval, classification, and clustering -- on 113,148 WIPO patents for assistive technology (46,069 citation queries) and on an external DAPFAM dataset, we find that two results cast doubt on the prevailing wisdom. (i) The optimal fine-tuning recipe depends on the downstream task: cross-sectional alignment (recipe R3) provides the largest improvements to retrieval performance (+7.1% nDCG@10), whereas a combined signal recipe (recipe R4) is better suited to classification (+7.1 F1) and clustering (+10.9 V-measure); a matched data control confirms that differences in training dataset size are not a contributing factor. (ii) Single-landscape fine-tuning hampers cross-landscape information retrieval: fine-tuning on one landscape significantly degrades cross-domain retrieval for 5 of 8 model-recipe combinations on the DAPFAM corpus, with the stronger zero-shot models suffering most. While within-family scaling is consistent (Qwen3 0.6B->4B->8B; Llama-Nemotron 1B->8B), cross-family scaling is erratic; the 12B KaLM-Gemma3 is ranked 8th on TAC retrieval performance, following prefix modification. Title+Abstract+Claims is the ubiquitous best text view, and all models suffer from a 55-65% gap between IN and OUT-of-domain performance which cannot be mitigated by hybrid BM25-dense fusion. Code and evaluation framework are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。