How to construct a self learning process for an ensemble model?

Question

I have two different (regression) models spitting out predictions on a daily basis for the same dependent variable. My intention is to assign weights to those two predictions and calculate a weighted average. To this end, I have developed a simple system where the MSEs of the models are calculated and used as weights everyday. As a result, the higher the MSE a model has, the lower the weight it is assigned to the model. However, this is a very lame approach and I haven't observed any improvement compared to taking a simple average of the predictions. So what are the ways that I can use to assign weights to those predictions dynamically. That is, I want to update the weights everyday. From where should I start?

Note: I am aware that with ensemble models, I can get the weights automatically (i.e. H2O stacked ensemble). However, I am not allowed to retrain the predictions everyday. So applying an ensemble method and retrain it everyday is not applicable in my case.

Thanks in advance!

Answer 1

It is not much more sofisticated than what you thought, but you can try training a linear regression (in the online setting, by stochastic gradient descent) whose inputs are the predictions spit by your models and the output is the real value of the predicted variable. In this case, you wouldn't need to retrain with the whole dataset.

In fact, I think that all online machine learning techniques can be applied in this setting.