My dataset contains four features. All of the features are categorical. There are 150 categories in the value of 1st and 2nd features. There are 8 categories in the value of 3rd and 4th feature. I replaced the categories with numeric values and applied Random forest. However, performance is not still up to the mark.

What other Machine Learning Classification algorithms I can try?


1 Answer 1


What does performance is not up to the mark mean?

To optimize performance you first need to understand it.

A typical workflow is this:

1. Define a baseline performanc

E.g. predict values with a constant (average, mode) or pick randomly from a distribution and measure the accuracy of that

2. Compare your 1st draft model (in your case your RandomForest) to that baseline

If your 1st draft model isn't better than the baseline, especially a random baseline, you have some errors in your code, data, etc. Try to find and eliminate those first.

If it is better but not by much, you have top optimize the model (see next step). If it is better by much you are either a) done or b) not satisfied with the absolute performance in which case you have to optimize.

3. Optimize your model parameters

Now you start grid searching your model parameters to get every last bit of performance out of it to make sure the problem is with the model and not the parameters.

4. Beauty contest

If your performance is still subpar now you can try other algorithms but do not expect wonders from this step. If you already tried RandomForest maybe go for:

  • Boosted models like XGBoost
  • Naive Gauss models
  • ...

Fit them on the same data, do parameter optimization and compare results.

5. Back to the data

Most likely just picking another model did not help if the original performance was too far away from acceptable. Then you have to go back to the data, collect more, feature engineer, etc.

  • $\begingroup$ Thanks for your suggestions. Is it okay replacing categories with numerical values? Or I should use onehotencoder ? $\endgroup$ Apr 24, 2020 at 9:43
  • 1
    $\begingroup$ @AtishDipongkor-MVP there is some discussion about best-practices for encoding categorical features. I tend towards OHE as my standard procedure but label encoding (what you did) is also quite common. In my oppinion switching between encoding methods should not lead to DRASTIC differences but the only way to know is try. So if feasible try-out OHE and see if performs better. For further inspiration see this blog post: towardsdatascience.com/… $\endgroup$
    – Fnguyen
    Apr 24, 2020 at 9:58
  • 1
    $\begingroup$ @AtishDipongkor-MVP Also you should try to identify whether there are large clusters in features 1 & 2 (via PCA, etc.) to trim down on the number of feature levels and maybe even identify new features. $\endgroup$
    – Fnguyen
    Apr 24, 2020 at 10:06
  • 1
    $\begingroup$ Thanks for your time! $\endgroup$ Apr 24, 2020 at 10:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.