0
$\begingroup$

I'm working on simple machine learning problems and I trying to build a classifier that can differentiate between spam and non-spam SMS. I'm confused as to whether I need to generate the document-term matrix before splitting into test and train sets or should I generate the document-term matrix after splitting into test and train?

I tried it both ways and found that the accuracy is slightly higher when the I split the data before generating the document-term matrix. But to me, this makes no sense. Shouldn't the accuracy be the same? Does the order of these operations make any difference?

$\endgroup$
  • $\begingroup$ If you are creating the document-term matrix from the entire dataset (test + train), then you're right, when you make the matrix shouldn't affect the accuracy. But the exact train/test split will affect the accuracy, as will any random operations during the training. $\endgroup$ – shinvu Aug 11 at 13:22
  • $\begingroup$ To have a real experiment, you need to ensure that everything - except the order of the split and the matrix generation - is exactly the same in both cases. $\endgroup$ – shinvu Aug 11 at 13:22
0
$\begingroup$

I gave a lot of thought about the question. I agree with you. But the slight difference might come if there are any random variable operation happens during the training. What model are you using for training?

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.