Return to Question

Made content clearer, made it clearer to see the context of the problem, the problem itself and the question the user is asking.

Source Link

edit approved Aug 9, 2020 at 14:54

shepan6

1.5k
6
14

General Data Science Question on Features that Don't apply to all obversations Imputing features with NA values in classification task

I currently have a dataset where each observation is a person's traffic ticket history in a certain districtover districts. The

For each column is 1 if the person has received 1+ traffic violations in, which represents a district in 2018 and 0 if they have been to that district but have no traffic violations, otherwise NA. The goal is to rank:

1 represents that a person has received 1+ traffic violations in a district in 2018

0 if they have been to that district but have no traffic violations.

NA otherwise

GOAL: to (rank districts to) see which district should have more police presence due to increased traffic violation and also use the features to predict whether or not that person has 1+ traffic accidents in 2019.

The problem is thatPROBLEM: not all people have been to every district. I currently just encode the value to 0 if the person has never been to that district. But this should be a valid NA value. For example, it's weirdit seems illogical to rank a district if only one person in(in the dataset amongst all people) has been to that district.

QUESTION(S): How exactly should I handle this? I don't think imputing as 0 is the right call here.

Original Data:

PersonId DistA DistB DistC DistD DistE Accident19
1        0     1     1     0      NA     1
2        NA    0     0     0      1      0
3        0     1     1     0      NA     1
4        1     0     0     0      NA     0

Imputed Data:

PersonId DistA DistB DistC DistD DistE Accident19
1        0     1     1     0      0     1
2        0     0     0     0      1     0
3        0     1     1     0      0     1
4        1     0     0     0      0     0

Many thanks in advance!

General Data Science Question on Features that Don't apply to all obversations

I have a dataset where each observation is a person's traffic ticket history in a certain district. The column is 1 if the person has received 1+ traffic violations in a district in 2018 and 0 if they have been to that district but have no traffic violations, otherwise NA. The goal is to rank districts to see which district should have more police presence due to increased traffic violation and also use the features to predict whether or not that person has 1+ accidents in 2019.

The problem is that not all people have been to every district. I currently just encode the value to 0 if the person has never been to that district. But this should be a valid NA value. For example, it's weird to rank a district if only one person in the dataset amongst all people has been to that district. How exactly should I handle this? I don't think imputing as 0 is the right call here.

Original Data:

PersonId DistA DistB DistC DistD DistE Accident19
1        0     1     1     0      NA     1
2        NA    0     0     0      1      0
3        0     1     1     0      NA     1
4        1     0     0     0      NA     0

Imputed Data:

PersonId DistA DistB DistC DistD DistE Accident19
1        0     1     1     0      0     1
2        0     0     0     0      1     0
3        0     1     1     0      0     1
4        1     0     0     0      0     0

Imputing features with NA values in classification task

I currently have a dataset where each observation is a person's traffic ticket history over districts.

For each column, which represents a district:

1 represents that a person has received 1+ traffic violations in a district in 2018

0 if they have been to that district but have no traffic violations.

NA otherwise

PROBLEM: not all people have been to every district. I currently just encode the value to 0 if the person has never been to that district. But this should be a valid NA value. For example, it seems illogical to rank a district if only one person (in the dataset) has been to that district.

QUESTION(S): How exactly should I handle this? I don't think imputing as 0 is the right call here.

Original Data:

PersonId DistA DistB DistC DistD DistE Accident19
1        0     1     1     0      NA     1
2        NA    0     0     0      1      0
3        0     1     1     0      NA     1
4        1     0     0     0      NA     0

Imputed Data:

PersonId DistA DistB DistC DistD DistE Accident19
1        0     1     1     0      0     1
2        0     0     0     0      1     0
3        0     1     1     0      0     1
4        1     0     0     0      0     0

Many thanks in advance!

Source Link

asked Aug 9, 2020 at 14:03

Eisen

General Data Science Question on Features that Don't apply to all obversations

Original Data:

PersonId DistA DistB DistC DistD DistE Accident19
1        0     1     1     0      NA     1
2        NA    0     0     0      1      0
3        0     1     1     0      NA     1
4        1     0     0     0      NA     0

Imputed Data:

PersonId DistA DistB DistC DistD DistE Accident19
1        0     1     1     0      0     1
2        0     0     0     0      1     0
3        0     1     1     0      0     1
4        1     0     0     0      0     0

machine-learning classification