0
$\begingroup$

I have several nominal variables which I've encoded using the LabelEncoder() function. Now I want to replace the encoded values in the place of the raw datas of the features in the dataframe. I tried df.replace() but no luck. Below is the code snippet.

nominal_values = [
    'HouseStyle','ExterQual','BsmtQual','BsmtExposure','BsmtFinType1',
    'KitchenQual','GarageType','GarageFinish','GarageQual','GarageCond']

One of the column "KitchenQual" has values as follows:

{kitchenqual :['Ex','Gd','TA','Fa','Po']} representing "Excellent","Good","Typical/Average","Fair","Poor" respectively.

Another Feature "BsmtFinType1" has values as follows:

{BsmtFinType1: ['GLQ','ALQ','BLQ','Rec','LwQ','Unf','NA']} represeting

"Good Living Quarters","Average Liv. Quat.","Below Avg.Liv.Quat,","Avg. rec room", "Low Quality", "Unfinished","No Basement" respectively.

from sklearn.preprocessing import LabelEncoder

lbe = LabelEncoder()
for noms in nominal_values:
    encode = lbe.fit_transform(cat_var[noms])
    cat_var.replace(to_replace=cat_var[noms],
                    value=pd.Series(encode),inplace=True)

I used pd.Series(encode) since the replace function will support only one of the following data structures

  • Scalar
  • dict
  • Series
$\endgroup$
3
  • 1
    $\begingroup$ Hi. If you want help with code, you should post a minimal working example illustrating your problem in as few lines of code as possible, not just a snippet. $\endgroup$
    – Miguel
    Dec 29, 2017 at 7:42
  • $\begingroup$ Hi @Miguel I have updated the question with the sample of feature values available for the same in the dataset. Please go through and guide me in where am I missing. Sorry for not giving out this information beforehand $\endgroup$
    – Vignesh
    Dec 31, 2017 at 11:35
  • $\begingroup$ What about cat_var[noms] = encode (or cat_var[noms] = pd.Series(encode) but I'm sure you just just use the list instead of casting as a Series) $\endgroup$
    – Dan
    Jan 4, 2018 at 18:47

1 Answer 1

0
$\begingroup$

I don't know what is wrong with your code but this worked fine for me-

cat_var.replace(to_replace=cat_var[noms].tolist(),value=encode,inplace=True)

Instead of converting encoded values to series, convert series to list using tolist().

$\endgroup$
5
  • $\begingroup$ Thank you so much @Ankit. It worked for me. But, Now there is a new issue. The Label Encoder does the job by sorting the values of the features alphabetically. Ex: For "KitchenQual", the encoded values returned are as follows: Ex:0; Fa:1; Gd:2; Po:3; TA:4 But, we can clearly say that the values we require are as follows: Po: 0 OR 1; Fa: 1 OR 2; Gd: 2 OR 3; TA: 3 OR 4; Ex: 4 OR 5; As of now, I've used the solution as follows: cat_var.KitchenQual.replace(to_replace=sorted(list(set(cat_var['KitchenQual'].values)),reverse=True),value=[3,4,2,5],inplace=True) Please help with a soln. $\endgroup$
    – Vignesh
    Jan 5, 2018 at 10:01
  • $\begingroup$ Due to the above-said issue, I am currently not able to use the Label encoder for encoding the nominal variables and I've used the mentioned code for all of the features and encoded manually $\endgroup$
    – Vignesh
    Jan 5, 2018 at 10:08
  • $\begingroup$ I dived deeper into this and found that at backend it basically uses np.unique to return the encoded values. [link] docs.scipy.org/doc/numpy-1.13.0/reference/generated/… This function,(np.unique), by default, sorts the input array so in your example you will always get Ex:0, Fa:1 etc. The solution (you know) is to sort the list in reverse order. $\endgroup$
    – Ankit Seth
    Jan 5, 2018 at 11:28
  • $\begingroup$ But the Lable encoder fit and transform methods doesn't take any sorting value as a parameter. I tried finding out by anyway would it be possible to make the label encoder to use the custom sort, but in vain. I am still struggling to get a permanent solution for that. $\endgroup$
    – Vignesh
    Jan 10, 2018 at 20:32
  • $\begingroup$ You can have a look at this- contrib.scikit-learn.org/categorical-encoding/ordinal.html $\endgroup$
    – Ankit Seth
    Jan 12, 2018 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.