I would like to understand more how different models (NN and RF specifically, but any other as well) consider interaction between features in tabular data?

For example, can the model figure out while training that "while feature 1 may not be directly correlated to the response, when feature 1 is low, feature 2 works really well at predicting the response". So the trained model would use the interaction between the two features and weight them accordingly, ie. weight feature 2 higher when feature 1 is low in the example.


Neural Networks (NN) model the feature interaction through the non-linear weighting of hidden nodes.

Random Forest (RF) are tree-based models. Tree-based learn feature interaction through recursive, conditional splitting. In the example you mention, a tree-based model would learn to split on feature 1 and then split on feature 2.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.