0
$\begingroup$

This is my first NLP project. I'm trying to use SMOTE for a classifier with 14 classes. I need to convert the classes into an array before using SMOTE. I tried using MultiLinearBinarizer but it does not seem to be working. From the stack trace, it seems like everything is getting converted. Do I need to convert something else to an array? How would I do that?

from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.preprocessing import MultiLabelBinarizer

nb = Pipeline([('vect', CountVectorizer()),
               ('tfidf', TfidfTransformer()),
               ('clf', MultinomialNB()),
              ])
nb.fit(X_train, y_train)


mlb = MultiLabelBinarizer()
print(mlb.fit_transform(df["Technique"].str.split(",")))
print(mlb.classes_)

import imblearn 
from imblearn.over_sampling import SMOTE 

smote = SMOTE('minority')

x_sm, y_sm = smote.fit_sample(X_train, y_train)
#print(x_sm.shape, y_sm.shape)
pd.DataFrame(x_sm.todense(), columns=tv.get_feature_names())

I'm getting the error ValueError: could not convert string to float: 'left left center'

Here is the stack trace

[[1 0 0 ... 0 0 0]
 [1 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]
 ...
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]]
['Appeal_to_Authority' 'Appeal_to_fear-prejudice' 'Bandwagon'
 'Black-and-White_Fallacy' 'Causal_Oversimplification' 'Doubt'
 'Exaggeration' 'Flag-Waving' 'Labeling' 'Loaded_Language' 'Minimisation'
 'Name_Calling' 'Red_Herring' 'Reductio_ad_hitlerum' 'Repetition'
 'Slogans' 'Straw_Men' 'Thought-terminating_Cliches' 'Whataboutism']
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-45-68681190410f> in <module>()
     10 smote = SMOTE('minority')
     11 
---> 12 x_sm, y_sm = smote.fit_sample(X_train, y_train)
     13 #print(x_sm.shape, y_sm.shape)
     14 pd.DataFrame(x_sm.todense(), columns=tv.get_feature_names())

8 frames
/usr/local/lib/python3.6/dist-packages/numpy/core/_asarray.py in asarray(a, dtype, order)
     83 
     84     """
---> 85     return array(a, dtype, copy=False, order=order)
     86 
     87 

ValueError: could not convert string to float: 'left left center'
```
$\endgroup$
0
$\begingroup$

You can't fit X_train into y_train without encoding. Try something like this for features:

from sklearn.preprocessing import OneHotEncoder
enc = OneHotEncoder(handle_unknown='ignore')
enc.fit(X)

and label encoding for labels.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ It is working now, thank you! $\endgroup$ – Sanglang Mar 31 at 5:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.