1
$\begingroup$

I am trying to predict a categorial variable given a set of input variables, which are also categorical. Both the target and features variables only take the class values [below mean, mean, above mean].

I have used one-hot encoding on both the target and feature vectors, but I am not sure if this is correct.

If I have 2 feature vectors I get 2x3 columns after one-hot encoding. For example, the first instance may be:

X[0] = [0,1,0,    1,0,0]

indicating the first feature has mean value ([0,1,0]) and the second feature is below mean ([1,0,0]), with the corresponding target

y[0] = [1,0,0]

indicating the target is below mean.

With the encoded X and y, I have tried to do this:

svm_clf = SVC()
svm_clf.fit(X_train, y_train)
y_pred = svm_clf.predict(X_test)

but it gives me 'ValueError("bad input shape {0}".format(shape))'

I have given the input and output below for the real situation, where the input has 9 features (so 9x3 columns after encoding). The error message indicates there is a problem with the 'y' vector.

Should I be one-hot encoding the target?

How can I tell the classifier that the output can only take a "single value" i.e. the 3 columns in the target are not independent as they all belong to the one variable. For example, the output for a given instance cannot be [1,1,0] as this indicates it is both "below mean" and "mean" which is not possible.

I have also tried a random forest classifier which ran OK, but the results were not plausible, so I assume I am doing something wrong,

Input features:

X_train type: and shape: (872, 27) and head(5):

[[ 0.  1.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.]
 [ 0.  1.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.]
 [ 0.  1.  0.  0.  1.  0.  0.  1.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.]
 [ 1.  0.  0.  0.  0.  1.  1.  0.  0.  0.  0.  1.  1.  0.  0.  0.  0.  1.  0.  0.  1.  0.  0.  1.  1.  0.  0.]
 [ 1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  0.  0.  1.  0.  1.  0.  1.  0.  0.]]

Target:

y_train type: and shape: (872, 3) and head(5):

[[ 0.  0.  1.]
 [ 0.  0.  1.]
 [ 1.  0.  0.]
 [ 0.  0.  1.]
 [ 0.  1.  0.]]

Error message:

  File "analyse.py", line 287, in dummy_test
    svm_clf.fit(X_train, y_train)
  File "/usr/lib64/python3.6/site-packages/sklearn/svm/base.py", line 149, in fit
    X, y = check_X_y(X, y, dtype=np.float64, order='C', accept_sparse='csr')
  File "/usr/lib64/python3.6/site-packages/sklearn/utils/validation.py", line 578, in check_X_y
    y = column_or_1d(y, warn=True)
  File "/usr/lib64/python3.6/site-packages/sklearn/utils/validation.py", line 614, in column_or_1d
    raise ValueError("bad input shape {0}".format(shape))
  ValueError: bad input shape (872, 3)
$\endgroup$
  • $\begingroup$ I think this issue is specific to SVM implementation, which is usually a binary classifier. I don't even know whether sklearn's SVM can be used for your multiclass problem. The input and output structures look correct for a more general case. So I'm going to alter the tags and title to reflect those details. Feel free to roll back the edits if you disagree. $\endgroup$ – Neil Slater Mar 7 '18 at 8:08
1
$\begingroup$

See the svm documentation here. It does support multiclass problem. The only thing is that your y should be like this [0, 1, 2, 3]. If you do not use one-hot encoding and use the y label as it is, it might work.

$\endgroup$
  • $\begingroup$ Yes, solution was to use label encoding as suggested by @Biranjan. $\endgroup$ – John Mar 8 '18 at 0:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.