0
$\begingroup$

I used weka program to make a classification, first I used a dataset using the explorer and the seed value was equal to one, then I used the experimenter for the same dataset and the seed value was also equal to one and I got the same classification results. But the second time I used the same data set and the value of the seed is equal to two through the explorer, but the result obtained by the explorer is different from the result obtained by the experimenter, knowing that the value of the seed is also equal to two in the experimenter window.

What is the reason for this ?

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ Can you put what classification model you are using? Many classification model uses random seed, and each one of them react differently with seed changes. $\endgroup$
    – Yohanes Alfredo
    Nov 25, 2019 at 9:03
  • $\begingroup$ In both cases i used random forest with cross validation option, and fold =10 $\endgroup$
    – Ahmad Sarairah
    Nov 25, 2019 at 10:32
  • 1
    $\begingroup$ so there you have it, you have 3 sources of randomness(2 from random forest, you can try finding out where these two sources are) and then from cv(referring to the cv split arrangement). For the CV result the average loss or any metric would not changed by much. For the random forest the result could be quite fluctuating especially if you only use few trees. $\endgroup$
    – Yohanes Alfredo
    Nov 25, 2019 at 10:58
  • 2
    $\begingroup$ Seed is only meant to be to be used to make sure that any results would be consistent through reruns. This is very useful when passing models around other users e.g. sharing results between team members and validating claims(you claimed your result produce 80% but after rerun you only get 78%). The last issue is actually quite common within machine learning, where seed difference can change winning score for example in kaggle competition. tl;dr; just fixed it and don't think too much about it. $\endgroup$
    – Yohanes Alfredo
    Nov 25, 2019 at 12:39
  • 2
    $\begingroup$ Even if you improve your CV by "tuning" your seed your result would be entirely by chance and cannot guarantee generalizability. $\endgroup$
    – Yohanes Alfredo
    Nov 25, 2019 at 12:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.