提出可微训练目标DTO,精准优化故事反事实改写。
DTO: a Differentiable Training Objective for Effective Counterfactual Story Rewriting
- 用可微损失函数联合优化重写保真度与叙事一致性
- 在TimeTravel和ART数据集上超越最大似然基线
- 适合需要精细控制文本生成的场景
反事实故事重写是自然语言处理任务,要求修改现有故事以反映选定的替代事件,同时保留未受影响的情节元素和整体连贯性。尽管大语言模型在此任务上取得进展,但因修改通常规模小且高度局部化,传统最大似然训练容易忽略这些细微差别。基于强化学习的复杂方法则训练慢且难部署。为此,本文提出新型可微训练目标(DTO),直接优化反事实改写效果。通过端到端反向传播,使用完全可微的损失函数,同时奖励参考重写保真度和源叙事语义一致性。在TimeTravel和ART数据集上的实证评估表明,所提DTO方法优于最大似然基线和偏好学习方法,在所有评估指标上与两个先进大语言模型表现相当。结果验证了针对特定任务的可微目标对精细可控文本生成的有效性。
原文摘要 · Abstract (English)
Counterfactual story rewriting is a natural language processing task that requires updating an existing story to reflect a chosen alternative event, yet preserving all the unaffected storyline elements and overall coherence. While large language models have recently made remarkable progress on this task, it still remains challenging since the required modifications are typically very small in size and highly localized. As a consequence, models trained in a conventional manner with the maximum-likelihood training objective tend to overlook these nuances. At the same time, more sophisticated training approaches based on reinforcement learning are notoriously slow and difficult to set up. For these reasons, our paper proposes a novel, differentiable training objective (DTO) that directly optimizes for the requisite counterfactual improvements. In our approach, a transformer model is fine-tuned via end-to-end backpropagation against a fully differentiable loss function that jointly rewards (i) fidelity to the reference rewrite and (ii) semantic consistency with the source narrative. The empirical evaluation on the TimeTravel and ART datasets shows that the proposed DTO approach has been able to surpass a maximum-likelihood baseline and a preference-based approach, and perform competitively against two contemporary large language models in all evaluation metrics. These findings substantiate the effectiveness of task-specific differentiable objectives for nuanced, controlled text-generation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。