1
$\begingroup$

I have two different (regression) models spitting out predictions on a daily basis for the same dependent variable. My intention is to assign weights to those two predictions and calculate a weighted average. To this end, I have developed a simple system where the MSEs of the models are calculated and used as weights everyday. As a result, the higher the MSE a model has, the lower the weight it is assigned to the model. However, this is a very lame approach and I haven't observed any improvement compared to taking a simple average of the predictions. So what are the ways that I can use to assign weights to those predictions dynamically. That is, I want to update the weights everyday. From where should I start?

Note: I am aware that with ensemble models, I can get the weights automatically (i.e. H2O stacked ensemble). However, I am not allowed to retrain the predictions everyday. So applying an ensemble method and retrain it everyday is not applicable in my case.

Thanks in advance!

$\endgroup$
2
  • $\begingroup$ Is your problem a regression or a classification problem? $\endgroup$ Apr 26, 2018 at 11:21
  • $\begingroup$ It is a regression problem. I have added it in the explanation. Thanks for reminding it! $\endgroup$ Apr 26, 2018 at 11:27

1 Answer 1

1
$\begingroup$

It is not much more sofisticated than what you thought, but you can try training a linear regression (in the online setting, by stochastic gradient descent) whose inputs are the predictions spit by your models and the output is the real value of the predicted variable. In this case, you wouldn't need to retrain with the whole dataset.

In fact, I think that all online machine learning techniques can be applied in this setting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.