0
$\begingroup$

I have a unique dataset that has many columns and most columns contain both categorical and non-categorical data. For example, let's say that one column is attribute_1 and for observations that have data for attribute_1 the value can be between 100 and 1000. If an observation does not have data for attribute_1 then they are given a value between -4 and -1, where the value describes why they don't have data for this attribute.

How can I encode the categorical part of the columns while also applying feature scaling to the non-categorical part of the column? Would it make sense to split the column into two where one is the categorical and another column just for the non-categorical?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

If you provide more information about the detail of why there is no data for some data points that would make it easier.

That being said, I would split it into three columns as follows:

col_1: it includes 0 and 1 >>> 1 for those samples that have a value of -4 in the original column. For the rest of the samples: 0

col_2: it includes 0 and 1 >>> 1 for those samples that have a value of -1 in the original column. For the rest of the samples: 0

col_3: the real values of the original columns. If the value is -4 or -1, there are two options as follows: 1. Use 0 for those samples 2. If they are missing values replace them using one of the methodologies used for replacing the missing values.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the suggestion! Unfortunately, the dataset contains personal information provided by an outside source and I'm not even fully aware of what that attribute is specifically or what the categorical values indicate. $\endgroup$
    – user2113499
    May 29, 2020 at 15:54
  • $\begingroup$ In that case, you can use the above method. For col_3, replace them with zero. $\endgroup$
    – nimar
    May 29, 2020 at 15:55
2
$\begingroup$

Would it make sense to split the column into two where one is the categorical and another column just for the non-categorical?

Absolutely yes. Split it into more than one column, that's the way to go.

At that point, each column can receive its appropriate scaling.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.