通过噪声调控提升阿拉伯语儿童读物生成多样性，保持阅读等级准确
Noise Steering for Controlled Text Generation: Improving Diversity and Reading-Level Fidelity in Arabic Educational Story Generation
- 在模型内部表示层注入高斯噪声以提升生成多样性
- 噪声方法使故事多样性提升37%，且不降低阅读等级准确性
- 适合需要严格控制词汇和难度的教育内容生成场景
为阿拉伯语低年级阅读评估生成多样且符合教学要求的故事，需在词汇、阅读难度和叙事结构的严格约束下避免情节重复。本文研究在推理阶段对变压器模型内部表示注入校准的高斯扰动（噪声引导），作为无需训练的多样性提升方法，在五个小型阿拉伯语专用语言模型（7-9B参数）上进行评估。对比四种注入策略与高温采样基线，测量多样性、质量、约束遵守度及阅读年级水平。残差流噪声可稳定提升叙事多样性，几乎不损失质量或违反约束，并在所有模型中保持早期阅读等级。注意力熵噪声注入（AENI）稳定了原本不可靠的注意力-逻辑噪声，同时恢复了生成质量。高温采样则显著提高阅读等级并导致多个模型出现灾难性崩溃。结果表明，内部表示层扰动比输出层随机性更适合受控教育内容生成。
原文摘要 · Abstract (English)
Generating diverse, pedagogically valid stories for Arabic early-grade reading assessments requires balancing tight constraints on vocabulary, reading level, and narrative structure against the need to avoid repetitive plots that undermine assessment validity. We investigate noise steering, injecting calibrated Gaussian perturbations into the internal representations of transformer models at inference time, as a training-free diversity method evaluated across five small Arabic-centric language models (7-9B parameters). We compare four injection strategies against high-temperature sampling baselines, measuring diversity, quality, constraint adherence, and reading grade level. Residual stream noise consistently improves narrative diversity with minimal quality or constraint cost and preserves early-grade reading level across all Arabic-centric models. Attention entropy noise injection (AENI) stabilizes the otherwise unreliable attention-logit noise while recovering quality. High-temperature sampling inflates reading grade level and causes catastrophic collapse on several models. We find internal representation-level perturbation to be a more suitable diversity strategy than output-level stochasticity for constrained educational content generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。