I'm working with LightGBM on a large data set about 3M row and about 8 columns. When i started feature generation after every new feature i was measuring the RMSE of the model and if it was the same or slightly worse i was removing it so is that procedure right ? or i should do all the feature generation and then apply feature selection on them and why ?


You can add features as you create them and check if the scoring improves (and by how much). Always measure the score on test data so you can detect overfitting.

When you are done with feature engineering you can perform feature selection to reduce the number of features or understand their importances in the model.

| improve this answer | |
  • $\begingroup$ but if added a feature and saw that its importance is higher than a lot of base features however, it increased the RMSE from 0.85 to 0.86 should i remove it or keep it ? $\endgroup$ – Mohy Mohamed May 12 at 15:15
  • $\begingroup$ You should probably keep it (because of high importance) and remove another one. Here the score goes down, probably because of overfitting. $\endgroup$ – Rusoiba May 12 at 18:03
  • $\begingroup$ okay thanks a lot $\endgroup$ – Mohy Mohamed May 13 at 0:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.