I have a structured dataset containing (nominal) categorical variables encoded as labels, let's say a feature includes labels from 1 to 20. Some of the labels in that feature could just be errors, that should not be present in the dataset and are not known priorly.
I wonder if there is an encoding method for that feature, such that the effect of erroneous labels will be mitigated. One-hot encoding of the labels before a ML task will create a dimension for each label, that could lead noisy labels to have a more dominating effect on the dataset.
In case of feature hashing, it's not that easy to determine the number of output features for each variable, therefore I think it wouldn't be reasonable to proceed with it.
Would a compression method such as PCA after having one-hot matrix (sparse) work well in this case? The labels would be represented by continuous values, although this could lead to an information loss for the correct labels besides noisy labels. But eventually noisy labels will not take up a dimension in the dataset, which is better.
I also believe that applying Fourier compression on one-hot matrices considering them as black and white images would be overcomplicated and nonsense for a tabular feature that frequencies do not matter.
What approach should I follow?
