0
$\begingroup$

Let say I have a feature that may have one of 4 values, 1,2,3,4. I want to provide it as a NN input, what is proper way to do that?

I can map it like 1 -> -1.0 | 2 -> -0.3 | 3 -> 0.3 | 4 -> 1.0 , or something similar to have mean of 0.0 and std near 1.0. But in this example 1 is much different than 4 compared to difference of 3 and 4, and I don't want such discrimination cos 1 and 4 are equally different to me as 3 and 4.

Another way is to have 4 features where each of them relates to a class 1,2,3,4 and has value of 1 if initial feature has value that matches it, and has value of 0 if it doesn't match it.

Like this 1 -> [1,0,0,0], 2 -> [0,1,0,0], 3 -> [0,0,1,0], 4 -> [0,0,0,1] But I don't like here the fact that this feature gets to much weight, especially if it has many classes.

I was thinking to make separated layer just for this feature, do you have some better solution?

$\endgroup$
1
$\begingroup$

In your problem, the label is a categorical variable (you cannot infer relation between classes just from the label value) and not ordinal (value shows relation/distance between classes).

The solution that you propose:

1 -> [1,0,0,0], 2 -> [0,1,0,0], 3 -> [0,0,1,0], 4 -> [0,0,0,1]

is called One-Hot Encoding. This is one of the most popular ways of encoding the classes during preprocessing, in order to feed them to a classifier, thus I recommend you of doing so.

You mention that you are afraid that the

feature gets to much weight

because of the number of samples in the dataset. This is called class imbalance. A way of circumventing it is to pre-weight your samples for your classifier, please take a look at this method implementation provided by sklearn package.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Hi, thank you for your answer. But I didn't quite understood the part with class imbalance. As I am familiar with it, it is when one output label has higher frequency than the other, right? How is it correlated with my problem? My concern was that if I have f.e. 3 simple regression features and one categorical which I apply OHE to, that this categorical will have more weight (if I have 4 classes in it) Meaning when regularization is being applied it will calculate 4 weights of that one feature and 3 weights of the other 3 features, which I don't like $\endgroup$ – Djura Marinkov Jul 6 '18 at 7:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.