DIRECT提升大模型序列标注的准确与速度，通过优化训练和高效解码。
DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models
- 训练后用直接偏好优化强化人类偏好对齐
- 解码时固定格式并限制候选集，提升准确率
- 模板填充仅生成标签词，复用缓存加速推理
序列标注是细粒度信息抽取任务，现有基于大模型的方法存在领域对齐不足和推理效率低的问题。为此，我们提出DIRECT框架，通过训练阶段优化与推理阶段修正协同解决。具体而言，DIRECT在监督微调后进行直接偏好优化（DPO），增强任务与人类偏好的对齐；引入受控解码机制，强制输出固定格式并限制预测范围至候选集合。为进一步提升效率，采用模板填充机制，仅要求模型生成标签词，通过KV缓存复用前缀内容，减少冗余计算。在八个数据集上的实验表明，DIRECT相较于现有方法在性能和效率上均有显著提升。
原文摘要 · Abstract (English)
Sequence labeling is a fine-grained information extraction task, yet existing large language model-based approaches suffer from insufficient domain alignment and low inference efficiency. To address these issues, we propose DIRECT, a framework that addresses these issues through training-time optimization and inference-time rectification. Specifically, DIRECT performs Direct Preference Optimization (DPO) after supervised fine-tuning to strengthen task alignment with human preferences, and introduces a controlled decoding process that enforces fixed output formats and restricts predictions to candidate sets. To further improve efficiency, a template-filling mechanism requires the model to generate only label tokens while reusing prefixed content through the KV Cache, thus reducing redundant computation. Experimental results on eight datasets demonstrate that DIRECT achieves significant improvements in both performance and efficiency compared to existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。