利用专利内部结构构建对比学习正样本,提升专利表征质量。
Patent Representation Learning via Self-supervision
- 用不同章节组合生成正样本,替代传统随机丢弃法
- 在多个专利检索任务中优于基线,部分超越依赖引用的模型
- 适合专利分析、技术搜索等需要理解专利结构的场景
我们研究基于对比目标的自监督专利表征学习。标准基线通过独立的丢弃掩码对同一文本编码两次生成正样本,但在长而结构化的专利文档上需精细调参。我们发现仅靠丢弃训练可被温度和丢弃率调节显著增强,但最优配置依赖评估任务,且无法在标题-摘要检索与权利要求-披露检索间通用。为此提出混合丢弃-章节正样本策略:锚点为标题-摘要视图,正样本来自该视图的丢弃重编码,或同一专利的其他章节(如权利要求、摘要、背景、附图、说明书)。该方法利用专利内部结构作为训练信号,无需IPC标签、引用或相关性标注。在分级欧洲专利局检索、新提出的家族级专利检索基准DAPFAM以及IPC子类分类任务上评估,基于章节的正样本优于校准后的丢弃法和通用标题-摘要增强基线,性能媲美基于引用的专利编码器和通用嵌入模型,并在DAPFAM的跨域测试集上表现优异。额外的跨章节对齐诊断显示,章节对训练提升了同一发明的摘要、权利要求与说明书间的兼容性。结果表明,专利章节能有效作为自监督正样本视图,用于学习密集专利表示。
原文摘要 · Abstract (English)
We study self-supervised patent representation learning with contrastive objectives. A standard baseline constructs positives by encoding the same text twice under independent dropout masks, but applying this recipe to long, structured patent documents requires careful calibration. We show that dropout-only training can be substantially strengthened by tuning temperature and dropout rate, yet its best configuration is evaluation-dependent and does not transfer uniformly from title--abstract retrieval to claim-to-disclosure retrieval. We propose mixed dropout--section positives, a patent-specific view construction strategy in which the anchor is the title--abstract view and the positive is sampled either from a dropout re-encoding of the same view or from another section of the same patent, such as claims, summary, background, drawings, or description. This uses patent-internal structure as a training-time signal without IPC labels, citations, or relevance annotations. We evaluate on graded EPO search-report retrieval, DAPFAM, a recently proposed family-level patent retrieval benchmark, and IPC subclass classification. Section-based positives improve over calibrated dropout-only and generic title--abstract augmentation baselines, are competitive with citation-informed patent encoders and a general-purpose embedding model, and perform strongly on the out-of-domain split of DAPFAM. Additional cross-section alignment diagnostics show that section-pair training improves compatibility among abstracts, claims, and descriptions of the same invention. These results indicate that patent sections provide effective self-supervised positive views for learning dense patent representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。