0
$\begingroup$

Trained the same model twice with the same dataset, the same parameters (Epochs, Batch Size, Learning rate, etc..). But both trained model shows different train as well as test accuracy on the same dataset. (code is same for two models)

Why both models show different accuracy?

  1. Train the model

    Test accuracy = 87.98%

  2. Again Train the model

    Test accuracy = 67.18%

Improve this question
$\endgroup$
5
  • $\begingroup$ If the training score is different then the 2nd model is not trained fully. What is the data size for both the set and what is happening in 3rd, 4th time? $\endgroup$
    – 10xAI
    Commented Dec 30, 2020 at 15:02
  • $\begingroup$ Yes, the training score is different for both the trained models. Dataset size for training is 13,558 (9 class). I have tried 2 times. Let me check for the 3rd and 4th trials also. $\endgroup$
    – Rina
    Commented Dec 31, 2020 at 3:36
  • $\begingroup$ At the 3rd time model's test accuracy: 85.68% & for 4th time: 76.38% $\endgroup$
    – Rina
    Commented Dec 31, 2020 at 10:16
  • $\begingroup$ You could use random_state=42 when splitting the data. $\endgroup$
    – idkfa.bfg2
    Commented Jan 29, 2021 at 23:13
  • $\begingroup$ If you share the code you are using it will be more easy for us to help. $\endgroup$
    – hH1sG0n3
    Commented Nov 30, 2021 at 13:11

3 Answers 3

Reset to default
0
$\begingroup$

Somewhere in the code there should be some parameter that is initialised randomly, this is usually called the random seed. It could be the different initialisation of your neural network weights that is affecting the results, or maybe the k-fold being different if done at random. The extent of the difference between the performance suggests that you might not have enough data to reliably learn a good model, or your model is underspecified and gets stuck at different local minima. Consider, changing the learning rate and other hyperparameters and see if the effect stays.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ The random seed value is selected as 2 in the code. So, there are any such criteria for the selection of random seed value? $\endgroup$
    – Rina
    Commented Dec 30, 2020 at 10:00
0
$\begingroup$

Often the initial training is slow, because the model is stuck in a part of parameter space with low gradients. Then it is essentially taking a random walk, until it reaches an attractor.

You can often get a better shaped loss surface by regularizing - then training is less reliant on luck.

Improve this answer
$\endgroup$
0
$\begingroup$

If you're using Tensorflow / Keras make sure to set a random seed with:

tf.random.set_seed(42)

Improve this answer
$\endgroup$
1
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Commented Nov 10, 2022 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.