用大模型实现无需梯度的智能策略分类,提升金融与互联网场景下的决策效率。
Breaking the Gradient Barrier: Unveiling Large Language Models for Strategic Classification
- 基于上下文学习设计无梯度优化框架,模拟策略分类中的双层优化过程。
- 在真实与合成数据上验证,模型在大规模场景下兼具鲁棒性与高效性。
- 无需微调即可适应动态环境,适合高规模、快速变化的决策系统。
战略分类(SC)研究个体或实体如何主动调整自身特征以获得有利分类结果。然而,现有方法多基于线性模型或浅层神经网络,在面对金融与互联网领域日益增长的数据规模时,面临可扩展性与表达能力不足的问题。本文提出一种基于大语言模型的新型框架GLIM,采用无梯度策略,依托上下文学习机制,在自注意力前向传播中隐式模拟典型双层优化流程,涵盖特征操纵与决策规则优化。无需微调预训练模型,该方法具备低成本适应动态环境的优势。理论上,证明了其可支持预训练大模型应对广泛的战略性特征调整。通过在金融与互联网领域的真实与合成数据集上的实验验证,GLIM展现出良好的鲁棒性与效率,为大规模战略分类任务提供了有效解决方案。
原文摘要 · Abstract (English)
Strategic classification~(SC) explores how individuals or entities modify their features strategically to achieve favorable classification outcomes. However, existing SC methods, which are largely based on linear models or shallow neural networks, face significant limitations in terms of scalability and capacity when applied to real-world datasets with significantly increasing scale, especially in financial services and the internet sector. In this paper, we investigate how to leverage large language models to design a more scalable and efficient SC framework, especially in the case of growing individuals engaged with decision-making processes. Specifically, we introduce GLIM, a gradient-free SC method grounded in in-context learning. During the feed-forward process of self-attention, GLIM implicitly simulates the typical bi-level optimization process of SC, including both the feature manipulation and decision rule optimization. Without fine-tuning the LLMs, our proposed GLIM enjoys the advantage of cost-effective adaptation in dynamic strategic environments. Theoretically, we prove GLIM can support pre-trained LLMs to adapt to a broad range of strategic manipulations. We validate our approach through experiments with a collection of pre-trained LLMs on real-world and synthetic datasets in financial and internet domains, demonstrating that our GLIM exhibits both robustness and efficiency, and offering an effective solution for large-scale SC tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。