This is my first post here, so kindly pardon any commonplace errors.

So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each class has 1000 rows.

My routine procedure is to train a model, get some metrics like Accuracy, ROC score and Pickle the model.

The problem is, when i load the pickled model in another Google Colab model with the same dataset, containing the exact same test, train splits as before, i get wildly different values of accuracy and ROC

I am ensuring the exact same data splits by using the random_state variable. Also in the XGBoost classifier, i am using fixed values of random_state and seed, to ensure there is no randomness in the results.

I will attempt to exemplify my problem.

For instance, i trained a model in notebook 1. I got accuracy value of 83.0806 % and ROC score of 91.3732 %

Happy with the results, i pickle the trained model for future use.

I tried to test the reproducibility of my work by opening a new Google Colab notebook (notebook 2 ) and loading the exact same dataset, with all the pre-processing steps and data splits same as before.

I then load the pickled model, use the predict function;


However, this time, i get very different values of accuracy & ROC score. I get an accuracy of 95.8870 % and a ROC score of 97.8383 %

Please note that this is the exact same dataset as used in notebook 1

can someone please tell me why this is happening ? is this an issue in Google Colab ?

Kindly help me out. I will be grateful for any guidance someone can provide in this regard


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.