2
$\begingroup$

Suppose I am training an lstm model on a stock price data.

So for first iteration say I have trained it on 80% of data and then tested it on rest of the 20% data and got the rmse value.

Now after this does it makes sense to again train it the whole data before predicting the value?

Example i have data for aapl from 2010 to today and I have trained it on 2010 to 2020 and tested on from 2020 till today and got the rmse values.

Now before predicting the next day value does it makes sense to again train it on whole data set i.e. from 2010 till today?

Because what I have observed is that in testing is that initial predictions have less error than the farther ones so i thought maybe I should train it on whole data set before predicting the next day or week value given that I know the accuracy of model from testing earlier samples.

Does it sound good or it has drawbacks that I am not aware?

$\endgroup$
1

2 Answers 2

1
$\begingroup$

Yes, first you select the best model & measure its performance using training & test data, then fit this best model on the full data. We try to use as much data possible for better results. Refer this stackexchange answer where he explained this: https://stats.stackexchange.com/a/366288

and also this article explanining why use of train/validation/test splits and using full data to train final model: https://machinelearningmastery.com/train-final-machine-learning-model/

I hope this helps.

$\endgroup$
1
$\begingroup$

Absolutely not! This is a basic tenet of Machine Learning. Due to the stochastic nature of training an ML model it's important to test the final model to ensure it satisfies our key criteria.

The other answer cites an article that highlights the benefit of using cross-validation. However, the key argument is that once the "best" modelling parameters have been determined, there is no longer a need to test a model and all of the data can be used in training.

This misses a key point about model training. Just because you have the best parameters, it doesn't mean that particular model is acceptable. You can see this for yourself, take the same data and repeat the model creation process. Even if you use the exact same data on the same computer you get slightly different results.

The concept of a "Final Model" is a leap of faith that it maintains the characteristics of the previous models. You have to take a risk that it is equally good despite make two significant changes: you've increased the amount of data and also it's the first time the model has seen all the data simultaneously. There remains a risk of overfitting for example. Given a decent length of experience in this field will show best practice remains to always test your final model.

In terms of best practice when dealing with Time Series data is that the split should not be random. Instead the test data should be the most recent as that's closest to today.

Jeremy Howard covers this topic well in one of his videos on Deep Learning.

$\endgroup$
1
  • $\begingroup$ there's a third significant change: you're (probably) using a completely different set of initial weights. $\endgroup$
    – Jumboman
    Feb 6 at 10:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.