2
$\begingroup$
  • I am dealing with the feature in which only one category takes up about 90%, the instances of more than 30 other categories are sparse.

enter image description here

  • Is it reasonable to remove this feature before building an estimator? I've experimented with the case when including and excluding this feature, and the later case showed a slightly(very slightly...) better performance.

  • I am wondering if there is any machine learning principle, such as we should avoid 'human learning' and just let 'machine learning' do. What would be the best way to deal with this sort of features? I am currently using Scikit-learn.

I am a total beginner in data science, so any opinion will be valuable to me. :)

$\endgroup$
  • 2
    $\begingroup$ In this particular case, also consider lumping the rare categories together. $\endgroup$ – Ben Reiniger Sep 20 at 12:37
1
$\begingroup$

Is it reasonable to remove this feature before building an estimator? I've experimented with the case when including and excluding this feature, and the later case showed a slightly(very slightly...) better performance.

This, ultimately, is the best way to get an answer. Trust in cross-validation; if you're doing it robustly and it says that ditching the feature improves performance, then ditch the feature.

I am wondering if there is any machine learning principle, such as we should avoid 'human learning' and just let 'machine learning' do. What would be the best way to deal with this sort of features? I am currently using Scikit-learn.

Nope, human selection of features plays a part too. Some algorithms are resistant to junk features and can effectively ignore them, but they always present the potential for overfitting. The best way to deal with them is what you've done; try the model both with and without and see which performs better.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.