代码大模型调试能力随尝试次数指数衰减，新指标可预测最佳干预时机。
The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs
- 提出调试衰减指数DDI，量化调试失效时间点。
- 2-3次调试后能力损失60%-80%，但适时重启可恢复效能。
- 适合优化代码生成系统，提升调试效率的工程团队。
AI调试效果遵循可预测的指数衰减规律；尽管迭代调试对实际代码生成系统至关重要，但大多数模型在仅2-3次尝试后即损失60%-80%的调试能力。本文提出调试衰减指数（DDI），一个数学框架，用于量化调试失效时刻并预测干预时机。所提出的策略性重启方法，在调试过程的关键节点从利用转向探索，证明适时干预能有效挽救调试效能。DDI揭示了当前AI调试的根本局限，并提供了首个优化迭代代码生成策略的定量框架。
原文摘要 · Abstract (English)
The effectiveness of AI debugging follows a predictable exponential decay pattern; most models lose 60-80% of their debugging capability within just 2-3 attempts, despite iterative debugging being a critical capability for practical code generation systems. We introduce the Debugging Decay Index (DDI), a mathematical framework that quantifies when debugging becomes ineffective and predicts intervention points. Our strategic fresh start approach shifts from exploitation to exploration at strategic points in the debugging process, demonstrating that well-timed interventions can rescue the effectiveness of debugging. DDI reveals a fundamental limitation in current AI debugging and provides the first quantitative framework for optimising iterative code generation strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。