用符号推理提升大模型结构化预测的准确性和一致性
Mapping the Course for Prompt-based Structured Prediction
- 将大模型与组合推理结合，弥补自回归生成的不足
- 加入符号推理后，预测准确率和一致性显著提升
- 适合需要高可靠性的结构化任务，如知识抽取
大语言模型在无需任务微调的情况下表现出色，但易产生幻觉和不一致，尤其在复杂推理和结构化预测任务中表现不佳。本文提出将大模型与组合推理相结合，融合大模型的泛化能力与符号推理的结构一致性。通过大量实验发现，无论采用何种提示策略，引入符号推理均能显著提升预测的一致性和准确性。此外，使用结构化学习目标进行校准和微调，进一步提升了在挑战性任务上的性能，表明结构化学习在大模型时代依然具有重要价值。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated strong performance in a wide-range of language tasks without requiring task-specific fine-tuning. However, they remain prone to hallucinations and inconsistencies, and often struggle with complex reasoning, in part due to the limitations of autoregressive generation. We propose to address some of these issues, particularly for structured prediction, by combining LLMs with combinatorial inference to marry the predictive power of LLMs with the structural consistency provided by inference methods. We perform exhaustive experiments in an effort to understand which prompting strategies can best estimate confidence values for downstream symbolic inference, and find that, independent of prompting strategy, incorporating symbolic inference yields more consistent and accurate predictions than prompting alone. Finally, we show that calibration and fine-tuning with structured learning objectives further increases performance on challenging tasks, highlighting that structured learning remains valuable in the era of LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。