用乔伊斯的《芬尼根守夜》训练语言模型,实现突破性创意生成。
TinyTim: A Family of Language Models for Divergent Generation
- 基于《芬尼根守夜》文本微调,构建具备发散生成能力的语言模型家族。
- TinyTim-V1的词汇丰富度(Yule's K)是传统模型的20倍以上。
- 保留独特创作风格,牺牲基准性能以维持创造性输出。
在追求通用人工智能的过程中,模型开发主要依赖已知问题及其公认解法的大规模数据集,这必然导致系统趋于收敛,无法实现真正的创造性突破。受人类发散性思维启发,本文提出一个语言模型家族TinyTim,旨在作为更广泛系统中的发散生成源。这些模型通过在詹姆斯·乔伊斯《芬尼根守夜》的反简约文本上进行微调而创建。对无监督微调模型TinyTim-V1和新型指令微调版本TinyTim-V2的定量分析显示其具有强大的词汇创新能力;基础版V1的词汇丰富度(Yule's K)超过收敛基线模型的二十倍。该特性在家族中稳定存在:指令微调版V2保持统计上显著不同的表现,抵抗事实收敛,虽牺牲基准性能仍维持其核心生成风格。本工作建立了一种工程化方法,用于设计专用发散模型,与收敛系统结合后可重新定义问题,推动仅靠统计优化无法达到的突破。
原文摘要 · Abstract (English)
In the search for artificial general intelligence, model development and training has focused primarily on vast datasets of known problems and their accepted solutions. This process necessarily produces convergent systems which are fundamentally incapable of the conceptual reframing that is required for genuine creative breakthroughs. Inspired by the divergent cognitive processes that allow humans to make such creative leaps, our work introduces a family of language models, TinyTim, to serve as sources of divergent generation within broader systems. These models have been created by fine-tuning on the anti-parsimonious text of James Joyce's `Finnegans Wake'. Quantitative analysis of both an unsupervised fine-tuned model (TinyTim-V1) and a new instruction-tuned variant (TinyTim-V2) demonstrates a profound capacity for lexical invention; the foundational V1 model exhibits a Yule's K score for lexical richness over twenty times greater than that of convergent baselines. This trait is a stable property of the family, as the instruction-tuned V2 maintains a statistically distinct profile and resists factual convergence, sacrificing benchmark performance to preserve its core generative style. This work establishes a methodology for engineering specialized divergent models that, when paired with convergent systems, can reframe problems and force breakthroughs beyond the reach of statistical optimization alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。