LLM推理时会陷入自我强化的思维循环，该方法可动态打破僵局。
'Neural howlround' in large language models: a self-reinforcing bias phenomenon, and a dynamic attenuation solution
- 通过动态引入反向调节机制，打断模型自我强化的错误循环
- 使陷入僵局的AI系统恢复自适应推理能力
- 适合关注AI决策鲁棒性的研究人员和工程师
由大语言模型驱动的AI系统可能表现出一种我们称为‘神经回响’（neural howlround）的推理失败模式，即某些高权重输入在认知循环中持续主导，导致响应模式固化且难以纠正。本文探讨了该现象的内在机制，其与模型坍塌和偏倚显著性加权不同。我们提出一种基于衰减的修正机制，能动态引入反向调节，并可在‘锁定’状态的AI系统中恢复自适应推理能力。此外，我们还讨论了不当强化管理引发的其他相关效应。最后，我们概述了该缓解策略在提升真实世界决策任务中AI鲁棒性的潜在应用。
原文摘要 · Abstract (English)
Large language model (LLM)-driven AI systems may exhibit an inference failure mode we term `neural howlround,' a self-reinforcing cognitive loop where certain highly weighted inputs become dominant, leading to entrenched response patterns resistant to correction. This paper explores the mechanisms underlying this phenomenon, which is distinct from model collapse and biased salience weighting. We propose an attenuation-based correction mechanism that dynamically introduces counterbalancing adjustments and can restore adaptive reasoning, even in `locked-in' AI systems. Additionally, we discuss some other related effects arising from improperly managed reinforcement. Finally, we outline potential applications of this mitigation strategy for improving AI robustness in real-world decision-making tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。