0
$\begingroup$

I am new to machine learning and I have a couple of questions about a project.

So, I created a classifier using the MNIST data set for a ML project that I was working on. I augmented the data by shifting each original point 2 pixels up, down, left, and right (so I have 240,0000 new images in addition to the 60,000 original ones). I was wondering if I should bother retraining my model with a shuffled training set, since I think (please correct me if I am very wrong) that the order of the training data in a KNN classifier does not play a factor in the final model, since predictions are made on nearest neighbors and shuffling the dataset will not change these neighbors. *As a side note, I am using the classifier/model on user drawn images, and the one with the augmented data set (unsurprisingly) performs better than the model trained only with the original data.

However, I seem to be getting the same accuracy score (the exact same to the 4th decimal place) on my test set with my augmented data set (30,000 total images) as I did with my original data set (60,000 total images). Is this something to be concerned about/did I do something wrong?

Thanks so much for your help!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

If you are talking about K-nn algorithm per se, then, you do not have any advantage in shuffling your training set. Indeed, as you said, K-nn is just looking for the K nearest neighbors and does not care at all about the order of the samples, the algorithm will scan the entire training set for a single prediction, no matter what (unless you are using some efficient implementation like ball-tree or kd-tree).

Regarding the second point of your question, I am wondering what your pipeline is: are you augmenting the dataset before splitting in train-validation-test? If so, I think you are injecting some bias in the evaluation of you algorithm, because you are evaluating your algorithm in samples that do not exist. You should augment only the training set and then validate/test your algorithm on real data.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your reply, it definitely helped. To answer your question, my data is split into test and validation sets and only after it's split do I augment the data. So, no I am validating/testing the accuracy of my model only on my test data, and using all of my augmented training data in the creation of my model. I ran my model again, and I did actually get a (slightly) improved accuracy score on my test set (compared to the model trained on non-augmented data). $\endgroup$
    – Akshay Ghosh
    Dec 26, 2020 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.