I am doing a little project on the Chicago Crime Rate data set and I noticed that there are over 600,000
NA values, primarily in the location fields.
I feel that even though there are about 6 million rows (data from 2001 - present) that is a lot of data to drop (especially since the rows contain all other data like crime type, ward, date, location description, etc.)
Here are the columns and the number of NA's found in each column:
COMMUNITY_AREA ID CASE_NUMBER DATE BLOCK IUCR PRIMARY_TYPE DESCRIPTION 616029 0 0 0 0 0 0 0 LOCATION_DESCRIPTION ARREST DOMESTIC BEAT DISTRICT WARD FBI_CODE X_COORDINATE 0 0 0 0 47 614854 0 60921 Y_COORDINATE YEAR UPDATED_ON LATITUDE LONGITUDE LOCATION CRIME_TYPE COMMUNITY_NAME 60921 0 0 60921 60921 0 0 616120
When I look up RPubs for this project a lot of people are either dropping all rows with NA's or not even bothering to talk about the missing data, both solutions in my opinion are Not good solutions.
Part of why I don't want to drop all those rows is because each row is a valid crime, when I drop them I am now missing crimes, this contributes to my counts, and categories. And since the data does say what type of crime was committed I can include it in these counts.
Has anyone worked with this data before? Do you have a suggestion for handling the missing data? Or can I leave it in there, is there an issue with that?
I plan on doing a time series analysis on Crime Rate, however, there is no missing data in the crime field so I do not think it will effect it.