0
$\begingroup$

Have specific question about how to measure a model's performances:

  • Is it correct to have common samples between the train and the test sets?
  • Is it correct to have duplicates samples in the test set?
$\endgroup$
1
$\begingroup$

welcome to the forum. I dont know what you exactly mean with 1) „common samples“.

But train and test should have no common rows/observations. They should be two distinct sets.

Having duplicates in test is not a good idea since you want to test how well your model works on data not seen so far by the model. Duplicates in test shrink the number of unique test cases. So there is little use of duplicates in the test set.

$\endgroup$
2
  • $\begingroup$ Thanks for your answer. You get it right, common sample mean train and test aren't disjoint. Can you please detail why do they need to be disjoint? How can we interpret sklearn.accuracy_score (doesn't handle unique cases), if the test set isn't unique? $\endgroup$ – username-checkout Jun 5 '19 at 9:15
  • $\begingroup$ You train a model on one part of data (say row: 1,2,3). You test the trained model on another part (say row: 4,5). So you can check for external validity of what your model has learned. $\endgroup$ – Peter Jun 5 '19 at 9:20
0
$\begingroup$

The reason why you want to have a disjoint test and train set is to detect over fitting.
If your model overfit on your train set, he will be overfitted on the test set for the common samples.
So you may not detect overfitting (depends of the number of common samples).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.