通过优化嵌入训练策略，提升逻辑推理的精度与效率。
High Quality Embeddings for Horn Logic Reasoning
- 用三元组损失训练，设计更易产生重复词汇的锚点
- 动态平衡难易样本比例，训练中重点强化最难样本
- 在多个知识库上验证，适合需要精准推理的任务
神经网络可被训练以对逻辑推理者的选项进行排序，从而更高效地寻找答案。这一过程的关键是生成有效的嵌入，即逻辑语句的数值表示。本文提出并评估了几种生成嵌入的方法，以获得更好的下游结果。采用三元组损失训练，需提供包含锚点、正例和负例的样本。本文引入三个改进：生成更可能包含重复词汇的锚点；以确保简单、中等和困难样本均衡的方式生成正负例；在训练过程中周期性强调最难的样本。通过多项实验评估该方法，包括在不同知识库上对比多种嵌入，旨在识别出适合特定推理任务的嵌入特性。
原文摘要 · Abstract (English)
Neural networks can be trained to rank the choices made by logical reasoners, resulting in more efficient searches for answers. A key step in this process is creating useful embeddings, i.e., numeric representations of logical statements. This paper introduces and evaluates several approaches to creating embeddings that result in better downstream results. We train embeddings using triplet loss, which requires examples consisting of an anchor, a positive example, and a negative example. We introduce three ideas: generating anchors that are more likely to have repeated terms, generating positive and negative examples in a way that ensures a good balance between easy, medium, and hard examples, and periodically emphasizing the hardest examples during training. We conduct several experiments to evaluate this approach, including a comparison of different embeddings across different knowledge bases, in an attempt to identify what characteristics make an embedding well-suited to a particular reasoning task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。