Additional business rules in ensemble methods (RF, Boosted Trees)

Question

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees?

These additional business rules can be useful, for instance, to inform the model that such or such scenario is not possible (e.g. you can't predict this if you see that), which the model would have trouble learning from the training data because of noise.

I'm currently using LightGBM but I'd also be interested on how to achieve this in any other library that supports ensemble models.

Answer 1

The model learns the significant patterns that it finds in the data. If the model is fed with noisy/inconsistent data, it might learn the noise. The solution is simply to clean up the data, in order to avoid feeding noise to the model.

If I understand your scenario correctly, it's possible to detect whether an instance contradicts the business rules. Therefore it's possible to remove any such instance before training the model. This way the model is fed with clean data (all the instances satisfy the business rules).

Note that logically the same preprocessing should be applied on the test instances before applying the model, since the model is not supposed to predict anything for these instances.