How to structure data and model for multiclass classification in SVM?

Question

I am trying to predict a categorial variable given a set of input variables, which are also categorical. Both the target and features variables only take the class values [below mean, mean, above mean].

I have used one-hot encoding on both the target and feature vectors, but I am not sure if this is correct.

If I have 2 feature vectors I get 2x3 columns after one-hot encoding. For example, the first instance may be:

X[0] = [0,1,0,    1,0,0]

indicating the first feature has mean value ([0,1,0]) and the second feature is below mean ([1,0,0]), with the corresponding target

y[0] = [1,0,0]

indicating the target is below mean.

With the encoded X and y, I have tried to do this:

svm_clf = SVC()
svm_clf.fit(X_train, y_train)
y_pred = svm_clf.predict(X_test)

but it gives me 'ValueError("bad input shape {0}".format(shape))'

I have given the input and output below for the real situation, where the input has 9 features (so 9x3 columns after encoding). The error message indicates there is a problem with the 'y' vector.

Should I be one-hot encoding the target?

How can I tell the classifier that the output can only take a "single value" i.e. the 3 columns in the target are not independent as they all belong to the one variable. For example, the output for a given instance cannot be [1,1,0] as this indicates it is both "below mean" and "mean" which is not possible.

I have also tried a random forest classifier which ran OK, but the results were not plausible, so I assume I am doing something wrong,

Input features:

X_train type: and shape: (872, 27) and head(5):

[[ 0.  1.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.]
 [ 0.  1.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.]
 [ 0.  1.  0.  0.  1.  0.  0.  1.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.]
 [ 1.  0.  0.  0.  0.  1.  1.  0.  0.  0.  0.  1.  1.  0.  0.  0.  0.  1.  0.  0.  1.  0.  0.  1.  1.  0.  0.]
 [ 1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  0.  0.  1.  0.  1.  0.  1.  0.  0.]]

Target:

y_train type: and shape: (872, 3) and head(5):

[[ 0.  0.  1.]
 [ 0.  0.  1.]
 [ 1.  0.  0.]
 [ 0.  0.  1.]
 [ 0.  1.  0.]]

Error message:

  File "analyse.py", line 287, in dummy_test
    svm_clf.fit(X_train, y_train)
  File "/usr/lib64/python3.6/site-packages/sklearn/svm/base.py", line 149, in fit
    X, y = check_X_y(X, y, dtype=np.float64, order='C', accept_sparse='csr')
  File "/usr/lib64/python3.6/site-packages/sklearn/utils/validation.py", line 578, in check_X_y
    y = column_or_1d(y, warn=True)
  File "/usr/lib64/python3.6/site-packages/sklearn/utils/validation.py", line 614, in column_or_1d
    raise ValueError("bad input shape {0}".format(shape))
  ValueError: bad input shape (872, 3)

Answer 1

See the svm documentation here. It does support multiclass problem. The only thing is that your y should be like this [0, 1, 2, 3]. If you do not use one-hot encoding and use the y label as it is, it might work.