I have a data set of 1 million points and 30 features. The output variable has multiple classes (1 to $n$) but the problem I'm interested in is only concerned whether the output belongs to class 1 or otherwise. I therefore converted the problem into a binary classification problem. Due to the large amount of data, using sklearn's histogram-based gradient boosting classifier offered a model that was fast to train especially for hyperparameter tuning using cross-validation.
But I realized that my data set is imbalanced (10% 1's and 90% 0's). Hence, I tried other classifiers to improve the performance. HistGradientBoostingClassifier currently does not have the balance weights option.
I tried a simple logistic regression with balanced class weights. I tried various regularization parameters but the precision was around 15% and recall around 62%. It did not change much as I varied the parameter making me think that a logistic regression model is insufficient.
I tried imbalanced learn's balanced bagging classifier on top of the histogram-based gradient boosting classifier and was able to get the precision to 20% and recall to 72% without any hyperparameter tuning. However, I cannot tune the hyperparameters of the histogram-based gradient boosting classifier, only those of the balanced bagging classifier. Doing a randomized search cross validation takes long but reasonable compared to using the histogram-based gradient boosting classifier only.
I am also trying RandomForestClassifier with
max_depth=10, I was able to obtain a precision of 25% and recall of 45%. The problem with this approach is that this set of parameters alone took 4 minutes, much longer than any of the methods above. It is therefore hard to do hyperparameter tuning with RandomizedSearchCV as it would take a long time.
Right now, I am almost running out of ideas as to what to try next to obtain a high precision and recall score. I am tempted to try SVMs since they should be relatively faster to train and are nonlinear in the original feature space, hopefully more expressive than logistic regression. I am also going to try imbalanced learn's balanced random forest classifier in case the model can be trained reasonably fast.
Does anyone have other suggestions for my problem? I have 2 issues mainly: 1) speed of training so I can tune the hyperparameters and 2) increasing precision and recall for this imbalanced data set.