# SVM is taking too long for hyperparameter tuning

I am running SVM,Logistic Rregression and Random Forest on the credit card dataset. My training dataset has the shape (454491, 30). I performed 5-fold cross validation(which took more than an hour) with 'scoring' set as 'f1_weighted' and got the below results:

Logistic Regression Cross Validation F1 score: 0.9501631748525725
Random Forest Cross Validation F1 score: 0.9999383944188953
Support Vector Cross Validation F1 score: 0.9989703035983751


I chose SVM as Random Forest is prone to overfitting and SVM scored better than Logistic Regression.

I wanted to add regularization by hyperparameter tuning. I initially used GridSearchCV but it was taking a long time so I changed it to RandomizedSearchCV but even this is taking a very long time (around 4-5+ hours). According to the data description, the features have been scaled and PCA has been performed to preserve the private information. I have also used RobustScaler() on Amount and Time columns as they were not scaled.

tuned_parameters={"kernel":['rbf','sigmoid'],
'gamma':[1e-2, 1e-3, 1e-4, 1e-5],
'C':[0.001, 0.10, 0.0001,0.00001]}

tuned_SVM=RandomizedSearchCV(SVC(),tuned_parameters,cv=3,scoring='f1_weighted',random_state=12,verbose=15,n_jobs=-1)


Any suggestions on how to proceed?

• Are you training with a GPU? Also RandomSearch and GridSearch will be slower compared to a Bayesian optimizer. So I would recommend you use that instead here is one that you could use instead sigopt.com. – yudhiesh Sep 11 '20 at 14:50