通过特征工程与调参，SVR在加州房价预测中将准确率提升至0.723
Optimised Support Vector Regression for California Housing Price Prediction: The Critical Role of Feature Engineering and Hyperparameter Tuning
- 构建10个领域相关特征，结合随机搜索调优超参数
- 特征缩放使性能提升0.744，最终测试R²达0.723
- 适合关注传统模型优化的实践者和数据科学家
现有研究中，支持向量回归（SVR）在加州房价基准数据集上表现最差，此前报告的R²仅为0.60。本文探究该结果是否源于实验配置而非算法本身缺陷。通过系统化流程：从8个原始特征构造10个领域驱动衍生特征，借助集成特征重要性分析筛选关键变量，并在泄漏安全的scikit-learn Pipeline中采用三折交叉验证进行随机搜索以确定最优超参数。四阶段消融实验表明：仅特征缩放便使R²提升0.744（从-0.054增至0.690），特征工程贡献+0.026（达0.716），超参数调优带来+0.008（达0.723）。最终调优后的SVR测试R²为0.723，相比先前结果绝对提升0.123（约20%相对增益）。在十种模型比较中，其排名第四，低于XGBoost（0.832）、随机森林（0.814）和梯度提升（0.783），但显著优于简单基线。十折交叉验证均值R²为0.703（95%置信区间：[0.630, 0.775]），验证了良好泛化能力。性能提升主要归因于统一预处理流程中的正确特征缩放，特征工程与系统调参提供增量改进。
原文摘要 · Abstract (English)
In the recent literature, Support Vector Regression (SVR) has been cited as one of the weakest performers on the California Housing benchmark dataset, with Preethi et al. (2025)specifically ranking it last among the algorithms they tested, reporting an R2 of only 0.60. This paper examines whether the previously reported performance reflects experimental configuration choices rather than an inherent algorithmic limitation. A structured experimental workflow is applied: ten domain-motivated derived features are constructed from the eight raw inputs, an exploratory ensemble feature importance analysis identifies the most predictive candidates, and a randomised search over hyperparameter combinations with three-fold cross-validation selects the optimal SVR configuration within a leakage-safe scikit-learn Pipeline. A formal four-stage ablation study isolates the contribution of each component: scaling alone accounts for +0.744 in R2 (from -0.054 to 0.690), feature engineering adds +0.026 (to 0.716), and hyperparameter tuning contributes +0.008 (to 0.723). The resulting tuned SVR achieves a test R2 of 0.723, a 0.123-point absolute improvement over the previously reported SVR result (from 0.60 to 0.723, approximately 20% relative gain). In the ten-model comparison, the tuned SVR ranks fourth with R2 = 0.723, below XGBoost (0.832), Random Forest (0.814) and Gradient Boosting (0.783), while substantially outperforming simpler baselines. Ten-fold cross-validation yields a mean R2 of 0.703 (95% CI: [0.630, 0.775]), confirming robust generalisation. The observed improvement from R2 = 0.60 to R2 = 0.723 is associated primarily with proper feature scaling within a unified preprocessing pipeline, with domain-motivated feature engineering and systematic hyperparameter tuning, providing further incremental gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。