I have a dataset of 1600 rows and 28 columns. Only one column is partially complete with 1300 records. The rest is NaN. I did a value count of this columns and it has 84 different categories that are nominal. What is the best way to impute this column. I need to convert these in numbers impute it and then convert back. I understand that One-Hot encoding does not work in this case because of the high cardinality.

What is the best way to approach this problem?


You may want to look into Target Encoding as an example:


Another post from the forums:

Problem with converting string to dummy variables

  • $\begingroup$ as I understand from the first link target encoder is for binomial or continuous. unfortunately my data in that column is not either of them $\endgroup$
    – erialdo
    Sep 21 '20 at 22:18
  • $\begingroup$ Hi again @erialdo! The terminology is confusing! In the case of target encoding, the "target" is not the original data but what it is converted into. So, it will take categorical data and convert it to either a binomial or continuous target. Another link may help with seeing the details: maxhalford.github.io/blog/target-encoding $\endgroup$ Sep 22 '20 at 0:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.