1
$\begingroup$

I am in the middle of processing data for feature reduction (haven't decided the method yet). The data consists of a row of people and ~3000 column of attributes that correspond to that person and is a mix of numeric and categorical. For most of the columns, the values are 0 to 1 (least likely to likely) in terms of probability they have this attributes. However, I do have some attributes like age and income.

My questions are:

  1. For feature reduction/model training purposes - do I treat this age/income column as numeric where I normalize between 0 and 1 or categorical where each age is treated as category and one hot encoded. Does it make sense to normalize age between 0 and 1 considering most of the other columns have values between 0 and 1 that translate to probabilities.
  2. Does it makes sense for feature reduction to do PCA on continuous and MCA on categorical and combine them? Or is there a better approach?
Person Green Conscious Democrat Lives in Midwest Age
Person1 .89 .76 .21 34
Person2 .23 .45 .32 53
Person2 .57 .23 .89 22

Thank you!

$\endgroup$

1 Answer 1

1
$\begingroup$

For normalizing the data it depends heavily on what you want to do downstream. If you want to use machine learning models based on linear algebra, then normalizing makes sense because otherwise, you can skew your model towards the larger data values.

But if the downstream is a tree model or something then it doesn't really matter how you normalize the continuous and categorical features.

As for dimension reduction, the main purpose is to reduce the computation cost for the downstream task. So for the linear algebra-based models PCA would be the good call, as the model itself will be correlation based. Whereas if you are going to use a tree for the downstream, then a different criteria could be used, such as using correlation between features or entropy between the feature and target. As this would pre-emptively reduce the amount of computations. All of the techniques should be tailored to the specific purpose of the downstream, and will perform differently depending on the downstream task.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.