I have a dataset with parameters (features) a
,b
, c
, etc. We need to develop a model to predict a
(our target).
b
is correlated with a
significantly (85%) and I suspect linear dependence. c
is a measurement of b
in another depth, so it has a high correlation with b
and a good correlation with a
. Also, there are a bunch of other parameters (numerical features) whose dependence of a
is non-linear (from background knowledge and box plot analysis). These features don't have a high correlation with a
but have an effect on a
.
I am building a model to predict a
based on the knowledge above. I think that a multi-linear regression cannot be a good idea because of non-linear dependencies. On the other hand, b
is a strong predictor of a
and depends on a
linearly. Therefore, I need to ensemble both linear regression and other methods (e.g., random forest) in a single model to have advantages of both.