So this question is more theoretical, than a practical one. I got a dataframe with 4 classes of cars' body types (e.g. sedan, hatchback, etc.) and different characteristics (doors, seats, maximum speed, etc.). The goal is to build a model, which predicts class by means of provided features. The steps, which I've applied are the following:

  • Encode classes of body types into variables (0, 1, 2, 3
  • Check if classes are balanced and in case of imbalance correct this issue
  • Feature selection based on the results of Pearson, Chi-2, RFE, logistic regression and XGBoost
  • Applying k-fold cross-validation with XGBoost on the whole dataset.

What is the correct order of implementing steps from the second one and so on? Should I firstly balance classes, then pick features and then apply XGBoost? Furthermore, should I split dataset into train and test and only then apply CV or may I stack XGBoost with CV on the whole dataset?

UPD: the class distribution is below
1 0.512228
2 0.282609
0 0.118207
3 0.086957


Please find the steps below.

  1. Encode classes of body types into variables (0, 1, 2, 3).
  2. Remove the rows or fill the cells contains Nan values.
  3. Remove/Update cells have unrelated data like 10 doors etc.
  4. Remove outliers if present in column like speed.
  5. Make the data balanced if required.
  6. Apply right encoding techniques for every non numeric columns.
  7. Perform Standardization and Normalization.
  8. If you feel that you really have more columns then apply PCA with 95% variance to reduce the columns count.
  9. Apply Random Forest classifier with K-Fold cross validation in whole dataset(before apply PCA) which gives you good accuracy and the importance features.
  10. Please feel free to run different algorithms along with hyper-parameters tuning using GridSearchCV.
  • $\begingroup$ 1) Why remove the outliers? // 2) Why balance unbalanced data? $\endgroup$
    – Dave
    Oct 4 '21 at 21:26
  • $\begingroup$ Some ML algorithms are prone to outliers so it's better remove based on our use case. If we have unbalanced data( skewed class) then we won't get good accuracy in output. Example - Consider a mail spam classification problem, if we have 98% of NOT SPAM data and only 2% of SPAM data. Irrespective of training models we can directly get 98% instead of training models which will give less accuracy, $\endgroup$ Oct 7 '21 at 5:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.