we are using a multivariate time series dataset.I have model A and model B running on same dataset, they both has same train, valid and test sets(their sizes are same). But model A takes time stamps(time features) as extra input along with the features, model B just takes the features as input. Now how can I justify comparing the mse and mae of two mdels. if lets say model B has better mse and mae than model A? is this a right comparison?
1 Answer
$\begingroup$
$\endgroup$
1
You can justify comparing the mean squared error (MSE) and mean average error (MAE) of the two models; if model B has a better MSE and MAE, you could technically say that it may have better predictive performance (but it would also be a good idea to look at other metrics to gain an idea of predictive performance, too).
However, the important thing to say is that model A was trained on more features than model B was; but you can still say model B maybe has better predictive performance, even though it was trained on less features.
Hope this helps!
-
$\begingroup$ If we are running model A on GPU and Model B on CPU and training or inference time does not matter will this comparison still be acceptable. $\endgroup$– User1086Commented Aug 23 at 16:19