I have built nine meta models based on the model stacking principle, which I compare to a reference model for a number of time series. See the results below. The 22 base models that are trained on 70% of the training data produce forecasts on the last 30% of the training data on which the meta models are trained. These are then validated on the test set (last 20% of all data).
The Lin. SVR model's hyperparameters are set as follows, with other hyperparams. set to their default values:
C=0.1, fit_intercept=False, loss='squared_epsilon_insensitive', dual=False
I am surprised at how the Linear Support Vector Regression (SVR) models are greatly outperformed by the Linear Regression (LR) models. I don't know how this phenomenon can be explained, as the pitfalls of SVR discussed online are long runtimes (solved by using a linear kernel) and that the problem might not be linearly solved. The latter appears unlikely given how the LR model performs much better.
What could explain the behaviour of these SVR models?