1
$\begingroup$

I am working on classification data that have 9 classes and so many features. well, classes are categorical obviously as well as some features. I used the one-hot encoding technique to transform categorical data into numerical. The question is, can I instead of having 8 or 9 columns for each class, each with a binary value, can I instead have only one class label column, but with values from 0 to 8, where 0 represents the first class and 8 represents the last class? If this works, can I do the same on the features columns or not?

Thanks in advance...

Note: data in both class labels and the features are nominal, not ordinal.

$\endgroup$

2 Answers 2

0
$\begingroup$

I would discourage such a practice.

If you use this for your outcome variable, you are making a wrong distribution assumption. You risk getting nonsense predictions like being in between a cat and a crocodile. With a categorical (multinomial) distribution, you wind up with fractional predictions, yes, but those have reasonable interpretations as class membership probabilities. You can use those probabilities to make classifications. Better yet, you can assess the probability values themselves! (1) (2)

If you do this for the features, You are making up an ordering relationship (e.g., $cat<dog$) and allowing for arithmetic when there should not be any ability to add a cat and a dog and get a crocodile.

One place where this would be okay is if your software knows to interpret the integers as categories. However, all that means is that the software is smart enough to do the categorical encoding on its own. I suspect that this is more likely to be the case for $y$ than for features.

$\endgroup$
2
  • $\begingroup$ Thanks for your simple answer. As I have understood, doing this on nominal data is not possible, what I am not really sure I understood correctly is that can I do this type of encoding to the class column, or it will have the same problem as the feature columns? $\endgroup$ Commented Dec 28, 2021 at 18:11
  • $\begingroup$ @OmarOshiba What is a “class” column? $\endgroup$
    – Dave
    Commented Jan 3, 2022 at 5:52
0
$\begingroup$

Mapping categorical levels to integer values is commonly called feature hashing / hashing trick.

Feature hashing can be useful for certain machine learning algorithms (e.g., tree-based and neural networks). However, linear models (e.g., logistic regression) will be unable to learn the relationship between hashed features and target values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.