3
$\begingroup$

Say, given 10 independent features as input to my RF model, when feature 1 and feature 3 are 100 (or less), my model output must be 5 despite the values of the other features. How can I teach that relationship to my RF model - so it will always obey that?

I have tried adding repeated artificial data points to the training set where features 1 and 3 are less or equal to 100 and the output is always zero, but the RF model does not seem to understand/learn that relation just based on that.

I also tried to play with (1) number of trees (2) number of nodes and (3) max nodes without any success.

I am using the mlr library in R to create my RF model. Thanks.

Improve this question
$\endgroup$
2
  • $\begingroup$ If the output is always 5 for such scenario, can you subset the data into two sets: the feature1 == feature3 = 100 set, and the other. For the first set you simply output 5 as the prediction, then you train and build the other set normally? $\endgroup$
    – The Lyrist
    Commented Oct 24, 2018 at 15:47
  • $\begingroup$ This sounds like something that should be a new feature in your dataset. Why not create a new column that reflects the relationship and train the model on that? $\endgroup$
    – I_Play_With_Data
    Commented Oct 25, 2018 at 18:14

2 Answers 2

Reset to default
1
$\begingroup$

Machine learning algorithms are not to know, they predict. A tree-based ML algorithm will give you probabilities of being in each class, not whether it belongs to a particular class. By preparing your labels carefully; if your data has enough examples of which “feature 1 and 3 are less and equal to 100” and the result is 0, it is highly probable that your algorithm will predict its class with a strong accuracy. Obviously, the strength of the result depends on what your ML task is, what is your data, how it is structured and what your hyperparameter settings are. I do not know if there is another way to bend the forest in such a manner.

I know that is not the kind of an answer you expected, but I’ll be glad if I could help.

Improve this answer
$\endgroup$
0
$\begingroup$

One way to guarantee this relationship is to specify it explicitly through an if/else statement. E.g.: feature 1 == feature 3 == 100 => output = 5 else output = random forest predictions (you would train the random forest as "normal" is this instance).

A couple of other things:

  • changing the number of trees and/or nodes will not change the relationship between features - these are used to control accuracy and over-fitting.
  • Introducing artificial data points is risky as it may cause data imbalances within your training set.
  • creating a new column in the dataset which reflects the relationship might work, but it is not guaranteed.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.