6
$\begingroup$

So i have a PandasDataFrame with categorical variables in a column which i want to one hot encode i've used the following code from an ML udemy course

from sklearn.preprocessing import OneHotEncoder
onehotencoder=OneHotEncoder(categorical_features=[10])
Y= onehotencoder.fit_transform(X).toarray()

However i get the following error

ValueError: could not convert string to float:

A bit of information Y is converted to an object from a df using

Y=df.iloc[:,:].values

I want to oneHotencode the 10th column of y which contains string values. The type of Y in variable explorer is object and if execute

type(Y)

i get numpy.ndarray

I'm new to Pandas and sklearn and would really appreciate any help.

$\endgroup$
4
$\begingroup$

There is an easy way to use one hot encoding in pandas and you can read about it in the following link:

https://pandas.pydata.org/pandas-docs/stable/generated/pandas.get_dummies.html

The get_dummies method is easy and optimized for use in pandas Data Frames.

Good luck!

| improve this answer | |
$\endgroup$
4
$\begingroup$

It may be a late answer, but I got the same problem and below is the solution

# Don't use categorical_features=[10] in encoder init
from sklearn.preprocessing import OneHotEncoder
onehotencoder=OneHotEncoder()
Y= onehotencoder.fit_transform(X[:,[10]]).toarray()
| improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ This works with the newer versions of sklearn. (At the time of the question OneHotEncoder couldn't handle string data.) $\endgroup$ – Ben Reiniger Dec 17 '19 at 14:55
3
$\begingroup$

You need two steps:

  1. Use LabelEncoder to encode your string variables to integers
  2. Then use OneHotEncoder on your integer variables
| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ This was needed in older versions of sklearn (presumably including the version at the time of the answer), but is not longer necessary. $\endgroup$ – Ben Reiniger Dec 17 '19 at 14:54
1
$\begingroup$

You could just use a LabelBinarizer. Label binarizer will skip the two step process(converting string to integer and then integer to float) as mentioned by DontDivideByZero.

from sklearn.preprocessing import labelBinarizer
encoder = LabelBinarizer()
Y = encoder.fit_transform(X)

This way you will convert the entire X matrix, but later you can quite easily extract Y[10] which is the one hot encoded matrix that you are looking for.

| improve this answer | |
$\endgroup$
1
$\begingroup$

I was also facing the same issue. I tried every possible way to do and got into why it wasn't happening. Actually, earlier OneHotEncoding needed numerical value first (earlier we couldn't directly encode string type data to numerical using OneHotEncoding, so first we used to apply LabelEncoding first) and then we used to apply OneHotEncoding.

But Now, OneHotEncoding could directly work with String data types also, but here we need data to be of type -either a DATAFRAME or a 2D Array (Not Object type) example [50,1].

Now you can do so, this way:

from sklearn.preprocessing import OneHotEncoder
    onehotencoder = OneHotEncoder()
    X_new_enc= onehotencoder.fit_transform(X[:,[3]]).toarray()  #[String_Column Index]

OR you rather use get_dummies directly (pandas based)

X= pd.get_dummies(X)

Feel free to ask any doubts over this.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.