I'm trying to implement a Naive Bayes classifier, which uses either of hypercubic Parzen window or KNN to estimate a density function.

The data I'm using is Fashion MNIST. The steps I take are that first I zero center the vectorized data and divide it by its column-wise variance, and then I feed this to a PCA and get a 9 dimension vector.

As for the Bayes decisions, for each class of the dataset, I get its samples and train a density estimator to estimate the class conditional densities. Then I multiply it by the class probability, and by taking the index of the maximum probability for each sample, I assign a class to unseen data.

What I'm having a hard time digesting now, is that my Parzen based model works way worse than my KNN. I'm trying to see whether these two converge when I step by step grow my dataset while keeping the volume of hypercube and K of KNN related similar to this:

Parzen vs KNN

And while KNN gives decent results, Parzen is somewhat indifferent towards this.

Parzen and KNN when the number of samples grows

Any explanation will be highly appreciated!

  • $\begingroup$ one plausible reason is that Parzen windows are more prone to outliers, than k-NN estimation $\endgroup$
    – Nikos M.
    Jun 12, 2021 at 9:21


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.