Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have developed a RandomForest classification model and I am pretty satisfied with the results on the test set.
Now, my next step is to deploy the model. Before deployment, I want to remove the test set, so the model learns on the whole dataset. It is then going to be used to classify new unseen data.
How can I do this? I have used test_train_split to split the data into train and test, I have defined the random forest model and I have used it to fit(X_train, y_train).
It is common practice to retrain a final model before deploying. That final model will use the hyperparameters found through cross-validation and be trained on all the data (both train and test). Using all the data will result in the best estimate of the parameters.
There is no additional testing or evaluation of the final model.
You can just do it easily by removing test_train_split step. However, I don't think you should do that. The appearance of the test set is to make sure that your model is generalized enough to predict accurately the unseen data.
If you retrain the model on the whole dataset without splitting, it's basically a different model, not the one that you said you are satisfied with.
How can you ensure that the new model is not overfitting? You have no data to validate it. The only option for you is to deploy it and wait for the unseen data coming in and compare the result later on. It takes time and is not recommended.
