I have 2 categorical variables e.g. state and city. Missing are only in city. As opposed to throwing out all observations with missing values for city or throwing out city all together I was considering making a variable location that is a concat of the two columns e.g. state:california city:LA becomes California_LA & state: california city: Missing becomes california_None.

I am building a NN and I want to know what the cost of doing this is. Will I be required to do more computation because the unique values in the location variable will be much greater than state or city?


It depends on whether you retain the original columns or not. You are not providing any additional information to the NN either way, so it's just a matter of how many features to compute in each batch. That said, for the same reason, you might as well just use a value (e.g. -1 or 0) for the missing cities. Depending on your implementation language, this is probably quicker easier to implement in terms of data engineering, and will amount to the same thing in terms of results. Obviously the NN will need numerical values, so effectively you could just label encode the city feature and assign missing values to one of the labels.

There are some very minor considerations, such as if you combine state and city, you will end up with a larger label set for the single column, which, if you normalise to an appropriate range (e.g. 0 to 1), which NN's often benefit from and which can help to avoid exploding gradients, you will have a finer graduation than a column each for city and state. Depending on the size and complexity of your data, this could have an impact on the NN's performance (prediction-wise) but it is likely to be a very minor effect compared to all of the other decisions you'll need to make in the process.

| improve this answer | |
  • $\begingroup$ I am providing additional info because if i concat I have the info from city. if I do not I disregard all the city info. $\endgroup$ – Vince Miller Nov 9 '18 at 20:55
  • $\begingroup$ @VinceMiller yes, in those terms you would be providing additional info, but as you've said, this is undesirable. Clearly, the more features and samples you provide, the more data available to the model (desirable especially for an NN, of course) but the more computations required. However, in this instance, unless you have a very large number of samples, the impact on compute between dropping the city column or not will be negligible. I would definitely keep the city data. The question then is how to incorporate it, and that is the question I was trying to address in my answer. HTH. $\endgroup$ – Chris Nov 9 '18 at 21:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.