Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I've been tasked with cleaning a dataset with a "Drive Time" column that lists times taken to drive to a specific location in whole minutes. The values range from 3 to 180 minutes but there is another entry of "Over 3 Hours". The "Over 3 Hours" entries make up just 1.36% of the column so should I delete these, or should I change them to 180 minutes?
Many thanks in advance.
There are many ways to process a data column that is a combination of categorical and numeric variables.
One option is to create two derived features. The first feature would be categorical: "short drive time" or "long drive time". You can decide the threshold based on domain knowledge, "over 3 hours" would definitely be in the "long drive time" category. The second feature would be the numeric: "length of short drive time". Only the raw data that is numeric would be in that feature, "over 3 hours" entries would be dropped.
The goal of feature engineering is to provide as much information to the algorithm so it can learn patterns. It is best practice to avoid dropping or changing values while feature engineering.
