In a biometric recognition system, I have noticed that normalizing the extracted wavelet features leads to increasing the recognition accuracy. The classifier used is K-nearest neighbor (KNN), and mapping the features to normal distribution with zero mean and unit variance is the normalization step used. Why normally distributed wavelet features lead to better accuracy?
4
-
1$\begingroup$ Because when not normalizing your features before using a KNN model the model gives a higher relative weight to features with a larger range of values than to features with a smaller range of values. $\endgroup$– OxbowerceCommented Jan 1, 2022 at 21:10
-
1$\begingroup$ @Oxbowerce Could you please give a numeric example? $\endgroup$– NohaCommented Jan 1, 2022 at 21:25
-
2$\begingroup$ Let's take the extreme case where you have two features with values in the range 0-1000000 and 0-100 respectively. In that case the first feature will have a relatively larger effect when assigning new points when using Euclidean distance. See also this answer from stats.stackexchange which also contains a visual representation. $\endgroup$– OxbowerceCommented Jan 1, 2022 at 21:58
-
$\begingroup$ @Oxbowerce For two fingerprints from the same person, I think that their wavelet feature vectors will be in a similar scale, whereas the large difference in scale will be between two feature vectors from two fingerprints for different persons. And we need minimum distance between samples from the same person, and maximum distance between different persons. $\endgroup$– NohaCommented Jan 2, 2022 at 4:16
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
K-nearest neighbors algorithm (k-NN) performance is improved if all feature dimensions are on a similar scale. A similar scale for all dimensions allows each dimension to have a chance to contribute to being near.
The features do not need to be normally distributed to be on a similar scale.
answered Jan 2, 2022 at 1:25
-
$\begingroup$ But I have tried other normalization techniques like zero-mean unit variance normalization, and found that mapping the features to normal Gaussian distribution leads to better results. $\endgroup$– NohaCommented Jan 2, 2022 at 4:02
-
$\begingroup$ @Noha Please explain zero-mean unit variance normalization with an example. $\endgroup$ Commented Jan 23, 2022 at 13:51
-
$\begingroup$ @Brian Spiering a crisp and nice answer. $\endgroup$ Commented Jan 23, 2022 at 13:57