小模型加规则后处理，低成本低延迟实现高精度多标签结构化预测。
Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data
- 用少量数据微调小模型，结合确定性规则后处理提升结构化输出质量。
- 在53条未见数据上达成83.0%准确率，关键字段100%正确，推理仅需2秒。
- 适合对成本、速度和隐私要求高的垂直领域应用，如合规审查。
将前沿大语言模型用于特定领域的结构化评估任务会带来高昂的延迟、成本和数据隐私风险。我们提出一种混合框架：在仅219个精选样本上微调一个小型语言模型（LLaMA 3.1 8B，LoRA可训练参数占比2.05%），并搭配确定性规则后处理层。应用于对话转录的多标签合规评估（18个异构输出字段），系统实现100% JSON结构有效性，盲评中整体准确率达83.0%，最关键分类字段准确率为100%。在单张NVIDIA A100 GPU上推理耗时约2秒，比前沿API快2–5倍；每评估成本为0.013美元，较商用方案节省46%–76%。引入关键决策边界的硬负样本增强，并形式化神经-符号分解，证明经领域适配的小模型结合后处理，可达到前沿模型精度，同时显著降低运营成本、延迟与隐私风险。
原文摘要 · Abstract (English)
Deploying frontier large language models (LLMs) for domain-specific structured evaluation tasks incurs prohibitive latency, cost, and data-privacy overhead. We present a hybrid framework that fine-tunes a small language model (LLaMA 3.1 8B, 2.05% trainable parameters via LoRA) on only 219 curated examples and couples it with a deterministic rule-based postprocessing layer. Applied to multi-label compliance evaluation of conversational transcripts (18 heterogeneous output fields), our system achieves 100% JSON structural validity, 83.0% human-validated overall accuracy, and 100% accuracy on the most critical classification field in blind evaluation on 53 unseen production transcripts. On a single NVIDIA A100 GPU, inference completes in $\sim$2 seconds -- 2--5x faster than frontier APIs -- at USD 0.013 per evaluation versus USD 0.025--0.055 for proprietary alternatives, yielding 46--76% cost savings. We introduce targeted hard-negative augmentation for critical decision boundaries and formalize the hybrid neural-symbolic decomposition, demonstrating that domain-adapted small language models with postprocessing can match frontier model accuracy while dramatically reducing operational cost, latency, and privacy risk.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。