This is my first post here, so kindly pardon any commonplace errors.
So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each class has 1000 rows.
My routine procedure is to train a model, get some metrics like Accuracy, ROC score and Pickle the model.
The problem is, when i load the pickled model in another Google Colab model with the same dataset, containing the exact same test, train splits as before, i get wildly different values of accuracy and ROC
I am ensuring the exact same data splits by using the random_state variable. Also in the XGBoost classifier, i am using fixed values of random_state and seed, to ensure there is no randomness in the results.
I will attempt to exemplify my problem.
For instance, i trained a model in notebook 1. I got accuracy value of 83.0806 % and ROC score of 91.3732 %
Happy with the results, i pickle the trained model for future use.
I tried to test the reproducibility of my work by opening a new Google Colab notebook (notebook 2 ) and loading the exact same dataset, with all the pre-processing steps and data splits same as before.
I then load the pickled model, use the predict function;
However, this time, i get very different values of accuracy & ROC score. I get an accuracy of 95.8870 % and a ROC score of 97.8383 %
Please note that this is the exact same dataset as used in notebook 1
can someone please tell me why this is happening ? is this an issue in Google Colab ?
Kindly help me out. I will be grateful for any guidance someone can provide in this regard