Logic behind SMOTE-NC?

Question

In the SMOTE paper here, the authors present the logic for creating synthetic examples when some of the features are nominal and some are continuous (section 6.1, SMOTE-NC).

This example is provided:

$F_1$ = 1 2 3 A B C [Let this be the sample for which we are computing nearest neighbors] $F_2$ = 4 6 5 A D E $F_3$ = 3 5 6 A B K So, Euclidean Distance between $F_2$ and $F_1$ would be:

$Eucl$ = $\sqrt{(4-1)^2 + (6-2)^2 + (5-3)^2 + Med^2 + Med^2}$

Med is the median of the standard deviations of continuous features of the minority class. The median term is included twice for feature numbers $5: B→D$ and $6: C→E$, which differ for the two feature vectors: $F_1$ and $F_2$.

The paper lacks explanation about why the nominal features should be affected by the continuous ones.

Can anyone provide such explanation? Did I miss it in the paper?

Answer 1

I also thought about the very same question recently and I think I might have a possible explanation.

Since we need to calculate distances between k-nearest neighbors, we have to provide some synthetic value which denotes the difference between nominal features. As a matter of fact, it could be any value. For example, imagine you have your nominal features one-hot encoded. In this case the difference between two different nominal features would be 1.

However, you would also like to keep the calculated distances more or less on the same scale as if you would have by using only continuous features. I believe that using median of standard deviations of continuous features helps to achieve exactly that and is the reason why it was chosen as a measure of difference between nominal features.