2
$\begingroup$

While tuning the SVM classification model in Matlab, I came across the rng function in matlab in which seed (stabilizes the random shuffling of the data in the algorithm) is changed. When the function called is rng(1) then I am getting one accuracy value (99%). When it is changed to rng(2) then I am getting another value (57%). So there is a huge change in accuracy as visible. What does this mean? Am I training it wrong?

The train and test set correct rate (in %) that I am getting with different runs without changing rng are(train,test)
(96, 82.8)
(94.6, 95.3)
(96, 85.9)
(96, 90)
(95, 95)

$\endgroup$
  • 1
    $\begingroup$ What are the testing errors of both? Nothing can be said by just looking at the training errors. Maybe rng(1) is better than rng(2) or maybe it is overfitting. $\endgroup$ – Dawny33 Aug 9 '16 at 5:48
  • $\begingroup$ @Dawny33 So does this mean that there is overfitting? A good model should have the same accuracy for any rng right? $\endgroup$ – girl101 Aug 9 '16 at 5:52
0
$\begingroup$

The training errors in this dataset has a huge difference (99% vs 57%). So, maybe the one with the rng(1) split has overfitted your dataset.

So there is a huge change in accuracy as visible. What does this mean? Am I training it wrong?

The huge change might be due to overfitting. (Also, judge the model through validation curves, and then fit a model which balances the bias-variance plot.)

| improve this answer | |
$\endgroup$
  • $\begingroup$ made a small change in the question. Let me know whether still there is over-fitting or not $\endgroup$ – girl101 Aug 9 '16 at 7:15
  • 1
    $\begingroup$ @Rishika No, I don't think there's any overfitting there. But, that's without changing the rng value, right? $\endgroup$ – Dawny33 Aug 9 '16 at 7:17
  • $\begingroup$ yes. This is without changing rng $\endgroup$ – girl101 Aug 9 '16 at 7:22
  • 1
    $\begingroup$ @Rishika The model looks good to me, then :) $\endgroup$ – Dawny33 Aug 9 '16 at 7:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.