0
$\begingroup$

In working with a telecommunications data set with multiple categorical variables which all depend on Internet Service, a separate categorical variable, I ran into the problem where 'No Internet Service' is dummy encoded several times in my training data. See the below pie charts for an illustration of the problem.

I was wondering what can be done to avoid this repetition of 'No Internet Service' as a feature value across several features, as it increases the dimensionality of my training data significantly and makes all of my features very highly interdependent.

Feature selection after encoding helps, but often leaves just the dummy variable 'No Internet Service' when removing the categories which don't contribute to the label.

Pie Chart of Online Backup Feature

Pie Chart of Device Protection Feature

$\endgroup$

2 Answers 2

0
$\begingroup$

The increase in dimensionality due to 'No Internet Service' could be handled maybe with a 2 layer model. If 'No Internet Service' is true, you run the first level model with a handful of other variables (excluding those which depend on internet service availability). The second layer model runs only when internet service available, with all the variables.

$\endgroup$
0
$\begingroup$

You could try the following approach:

  1. Create Binary variables for "Online Backup" and "Device Protection" - Set as 1 for all "Yes" answers - and "1" for anything else (includes "No" and "No Internet service")
  2. Create a new Binary variable for "No Internet Service" consolidating the responses to "No Internet Service" across all other features.
$\endgroup$
2
  • 1
    $\begingroup$ This is exceptionally helpful, thank you so much for the suggestion $\endgroup$ Apr 16, 2021 at 18:31
  • $\begingroup$ Please accept as correct answer if you are satisfied $\endgroup$ Apr 17, 2021 at 4:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.