1
$\begingroup$

This is my first post here, so kindly pardon any commonplace errors.

So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each class has 1000 rows.

My routine procedure is to train a model, get some metrics like Accuracy, ROC score and Pickle the model.

The problem is, when i load the pickled model in another Google Colab model with the same dataset, containing the exact same test, train splits as before, i get wildly different values of accuracy and ROC

I am ensuring the exact same data splits by using the random_state variable. Also in the XGBoost classifier, i am using fixed values of random_state and seed, to ensure there is no randomness in the results.

I will attempt to exemplify my problem.

For instance, i trained a model in notebook 1. I got accuracy value of 83.0806 % and ROC score of 91.3732 %

Happy with the results, i pickle the trained model for future use.

I tried to test the reproducibility of my work by opening a new Google Colab notebook (notebook 2 ) and loading the exact same dataset, with all the pre-processing steps and data splits same as before.

I then load the pickled model, use the predict function;

loaded_model.predict(x_test)

However, this time, i get very different values of accuracy & ROC score. I get an accuracy of 95.8870 % and a ROC score of 97.8383 %

Please note that this is the exact same dataset as used in notebook 1

can someone please tell me why this is happening ? is this an issue in Google Colab ?

Kindly help me out. I will be grateful for any guidance someone can provide in this regard

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.