2
$\begingroup$

I am planning to do a simple classification with a linear SVM. One feature I have is another classification of some sort done previously. Can I just use this class feature as a 1-hot encoded array? So, e.g. for 3 different classes, I'd have 3 binary features being 0 or 1?

The problem I see is that this feature is not linear but binary. Will this pose a problem? And if yes, how can I somehow transform a binary feature into a linear one?

$\endgroup$
1
$\begingroup$

The quick answer is yes, you can use a liner SVM in presence of an encoded categorical variable.

Short explanation: The linearity of the model has nothing to do with the features of model, and actually linear feature doesn't mean anything. The linearity refers to the model, i.e. the equation that links the target to the features.

The equation

$$y = b*x + q$$

means that y is linear with respect to x, not that the variable x is linear. x is x thats it.

What you should check when dealing with linear models is if your data is linearly separable, or in other words if you can separate the different classes with a straight line. If that is not the case you are in trouble and should probably think of changing the model.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Okay, that actually makes a lot of sense now that you say it. E.g. an AND Gate is perfectly separatable by a linear classifier whereas an XOR Gate cannot be separated by a linear classifier. But both have binary "features". $\endgroup$ – Jodo Nov 15 '18 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.