0
$\begingroup$

Hello fellow data scientists.

I am new in this field, and I face to a problem, for what I need an advice. So I have data, where one column is product ID, and another which say from which city it originates.

So my question is now what do in the cases, when city value is empty? I think it is absolutely impossible, to guess the origin and fill, or fill with median.

so what is your advice?

Thank you very much

$\endgroup$

1 Answer 1

0
$\begingroup$

With little context given, we could deal with this in a very generic way. There are a couple ways to deal with categorical missing values:

  1. Leaving as it is. This depends on the proportion of missing values and which model you would be using (for example, using this method while planning to use regression model would make no sense).
  2. Getting rid of the entry. This likewise depends on the proportion of missing values. Assuming that this is your only feature, you wouldn't suffer much from losing other column data based on a single missing value, but regardless if the proportion of missing values is high, you would probably want to think otherwise.
  3. Filling in with a predictive value. Is this dataset contingent to another dataset that maybe shows a distribution of the origin cities? Is it reasonable that, based on the distribution, you replace the missing values with the city with the most frequency? If you have an access to an additional, related dataset, is the city column explained by other features?

There are obviously more sophisticated and detailed ways to tackle the missing value problem, but you would have to consider the context of the data, the proportion and distribution, and sometimes even the purpose of the data analysis.

$\endgroup$
1
  • $\begingroup$ Thank you, I guess in my case, just dropping them is the best solution, only only 5% of the data, which is around 800 000 records. $\endgroup$
    – Numenorean
    Commented Apr 19, 2022 at 10:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.