0
$\begingroup$

From my understanding in order to improve accuracy with ensemble models you need a wide range of independent ensemble methods. I was wondering whether using the ouput of a random forest model as one of the inputs for a neural network where the other input is the original data and the targets remain the same could improve the model? Why add extra complexity? basically the problem is a multi-dimensional regression problem and although the random forest gets a smaller MSE the neural network is bette at preservering some of the properties of the target labels. Therefore, I was wondering if by putting these two models together I would get a lower MSE while preserving some of the properties. Is it worth a shot? or will it just drastically overfit?

$\endgroup$
1
$\begingroup$

I suggest you test it by cross-validation to prevent overfitting. However, I guess because of a strong relationship between this variable and output your work will not be better. Instead, try to use an ensemble.

I read something about deep learning which in that way they use several models as a sequence and use results of previous models as inputs of the next models. Maybe if you continue this work it will be in some way a deep learning.

$\endgroup$
2
  • $\begingroup$ what do you mean by deep mining? $\endgroup$
    – Tank
    Jul 18 '18 at 9:10
  • $\begingroup$ Sorry, deep learning $\endgroup$
    – parvij
    Jul 18 '18 at 9:59
1
$\begingroup$

I'm working on prediction of time series and my stacked LSTMs alone does not capture well the trend and seasonality of my data. I decided to add features from a Holts Winter regression as an additional input and the result is way better. It slightly overfits but in my case that's not an issue.

In my case, I like the fact that my statistic (holts winter) model can capture well the global trend, and the network can find other non linear relations between datapoints. I guess it's a good idea to combine several models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.