I have a dataset with parameters (features)
c, etc. We need to develop a model to predict
a (our target).
b is correlated with
a significantly (85%) and I suspect linear dependence.
c is a measurement of
b in another depth, so it has a high correlation with
b and a good correlation with
a. Also, there are a bunch of other parameters (numerical features) whose dependence of
a is non-linear (from background knowledge and box plot analysis). These features don't have a high correlation with
a but have an effect on
I am building a model to predict
a based on the knowledge above. I think that a multi-linear regression cannot be a good idea because of non-linear dependencies. On the other hand,
b is a strong predictor of
a and depends on
a linearly. Therefore, I need to ensemble both linear regression and other methods (e.g., random forest) in a single model to have advantages of both.