1
$\begingroup$

I have built nine meta models based on the model stacking principle, which I compare to a reference model for a number of time series. See the results below. The 22 base models that are trained on 70% of the training data produce forecasts on the last 30% of the training data on which the meta models are trained. These are then validated on the test set (last 20% of all data).

The Lin. SVR model's hyperparameters are set as follows, with other hyperparams. set to their default values:
C=0.1, fit_intercept=False, loss='squared_epsilon_insensitive', dual=False

I am surprised at how the Linear Support Vector Regression (SVR) models are greatly outperformed by the Linear Regression (LR) models. I don't know how this phenomenon can be explained, as the pitfalls of SVR discussed online are long runtimes (solved by using a linear kernel) and that the problem might not be linearly solved. The latter appears unlikely given how the LR model performs much better. results

Looking at the actual forecasts of the Lin. SVR model, it is evident that there is a strong bias while the pattern of the target value appears captured moderately well. enter image description here

What could explain the behaviour of these SVR models?

Improve this question
$\endgroup$
6
  • $\begingroup$ Welcome to DataScienceSE. I don't see the "SVR with hyper-parameter tuning" in the graph? is it better than the basic SVR? If yes then this would explain it. In general my guess would be that there's something wrong with the way SVR is used or configured. $\endgroup$
    – Erwan
    Commented Jun 14, 2022 at 15:39
  • $\begingroup$ Hi @Erwan, the two SVR models are almost identical in performance and therefore partially overlap in the graph. Hyperparameter tuning therefore seems to have little effect. Any thoughts what could cause this? $\endgroup$
    – Tim Stack
    Commented Jun 14, 2022 at 21:14
  • $\begingroup$ I don't really know, it's strange especially when basic LR works well. Maybe lack of scaling the features? Or a problem with the scaling? SVR can be sensitive to this, but I doubt it would have this effect. $\endgroup$
    – Erwan
    Commented Jun 14, 2022 at 21:36
  • $\begingroup$ The data seen by all models is the same, and has been standardised using sklearn's scale() function $\endgroup$
    – Tim Stack
    Commented Jun 15, 2022 at 9:37
  • $\begingroup$ @Erwan I have amended my post with forecasts of the Lin. SVR model. These forecasts don't seem terrible, were it not for the fact that they seem to have a strong consistent negative bias. Would this warrant a new question? $\endgroup$
    – Tim Stack
    Commented Jun 15, 2022 at 9:49

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.