I understand data hygiene, which is probably the most basic feature engineering. That is making sure all your data is properly loaded, making sure
N/As are treated as a special value rather than a number between -1 and 1, and tagging your categorical values properly.
In the past I've done plenty of linear regression analysis. So feature engineering mainly concerned with:
- Getting features into the correct scale using log, exponent, power transformations
- Multiplying features: if you have height and width, multiply to make area
- Selecting features: remove features based on P value
But, for LightGBM (and Random Forest) it seems like the scale of the features doesn't matter because orderable items are ordered and then randomly bisected. Interactions of features don't matter because one of the weak classifiers should find it if it is important. And feature selection isn't important because if the effect is weak then those classifiers will be attenuated.
So, assuming you can't find more data to bring in, what feature engineering should be done with decision tree models?