0
$\begingroup$

I'm having a lot of issues using scikit learn recently and was hoping someone could help me with my problem. I can use other methods to ordinal encode but i want to figure this one out.

for i in range(len(ordinal_orders)):
   ord_en = OrdinalEncoder(categories = {0:ordinal_orders[i]})
   X_train.loc[:,ordinal_features[i]] = ord_en.fit_transform(X_train.loc[:,ordinal_features[i]].values.reshape(-1,1))

This works fine but when i try and apply this transformation to the test set i get an error.

for i in range(len(ordinal_orders)):
    ord_en = OrdinalEncoder(categories = [ordinal_orders[i]])
    X_test.loc[:,ordinal_features[i]] = ord_en.transform(X_test.loc[:,ordinal_features[i]].values.reshape(-1,1))

i get the value error

AttributeError                            Traceback (most recent call last)
<ipython-input-45-24cb27da6829> in <module>
      1 for i in range(len(ordinal_orders)):
      2     ord_en = OrdinalEncoder(categories = [ordinal_orders[i]])
----> 3     X_test.loc[:,ordinal_features[i]] = ord_en.transform(X_test.loc[:,ordinal_features[i]].values.reshape(-1,1))
      4 

/opt/anaconda3/lib/python3.8/site-packages/sklearn/preprocessing/_encoders.py in transform(self, X)
    698             Transformed input.
    699         """
--> 700         X_int, _ = self._transform(X)
    701         return X_int.astype(self.dtype, copy=False)
    702 

/opt/anaconda3/lib/python3.8/site-packages/sklearn/preprocessing/_encoders.py in _transform(self, X, handle_unknown)
    105         X_mask = np.ones((n_samples, n_features), dtype=np.bool)
    106 
--> 107         if n_features != len(self.categories_):
    108             raise ValueError(
    109                 "The number of features in X is different to the number of "

AttributeError: 'OrdinalEncoder' object has no attribute 'categories_'

The ordinal features and orders are

ordinal_features=['LotShape','ExterQual','ExterCond','BsmtQual','BsmtCond',
'BsmtExposure','BsmtFinType1','BsmtFinType2','HeatingQC','KitchenQual',
'FireplaceQu','GarageQual','GarageCond','GarageFinish','Fence','PoolQC']

ordinal_orders=[ 
    #LotShape 
['Reg','IR1' ,'IR2','IR3'], 
    #ExterQual 
['Fa','TA','Gd','Ex'], 
    #ExterCond
['Po','Fa','TA','Gd','Ex'],
    #BsmtQual 
['None','Fa','TA','Gd','Ex'], 
    #BsmtCond 
['None','Po','Fa','TA','Gd','Ex'], 
    #BsmtExposure 
['None','No','Mn','Av','Gd'], 
    #BsmtFinType1 
['None','Unf','LwQ', 'Rec','BLQ','ALQ' , 'GLQ' ],
    #BsmtFinType2 
['None','Unf','LwQ', 'Rec','BLQ','ALQ' , 'GLQ' ], 
    #HeatingQC 
['Po','Fa','TA','Gd','Ex'], 
    #KitchenQual 
['Fa','TA','Gd','Ex'],
    #FireplaceQu 
['None','Po','Fa','TA','Gd','Ex'], 
    #GarageQual 
['None','Po','Fa','TA','Gd','Ex'], 
    #GarageCond 
['None','Po','Fa','TA','Gd','Ex'], 
    #GarageFinish 
['None','Unf','RFn','Fin'], 
    #Fence 
['None','MnWw','GdWo','MnPrv','GdPrv'], 
    #PoolQC
['None','Fa','Gd','Ex']
]

train and test set is just a generic dataset from kaggle, theyre not an issue here. Any sort of clarification would be nice!

Improve this question
$\endgroup$
2
  • $\begingroup$ You are overriding your ord_enc object every time you fit_transform to a new column. You should to apply it to all the relevant columns at once, preferably with a ColumnTransformer. Then you override it again when you try to transform the test data, and now the object isn't even fitted! $\endgroup$
    – Ben Reiniger ♦
    Jan 11, 2021 at 15:49
  • $\begingroup$ I'm overriding it because the ord_enc is being replaced with the subsequent list being the new 'i' in ordinal_orders[i] in order to apply the according encoding at the fit_transform. However, i do now see the problem this leads towards when using the .transform. ColumnTransformer i go! $\endgroup$
    – Zèro
    Jan 11, 2021 at 16:54

2 Answers 2

Reset to default
0
$\begingroup$

It is really hard to figure out the logic behind what you are doing, it look odd

But assuming you are trying to apply a preprocessing step to a data frame I would go as follows:

from sklearn.compose import make_column_transformer
from sklearn.preprocessing import OrdinalEncoder


ordinal_features=['LotShape','ExterQual','ExterCond','BsmtQual','BsmtCond',
'BsmtExposure','BsmtFinType1','BsmtFinType2','HeatingQC','KitchenQual',
'FireplaceQu','GarageQual','GarageCond','GarageFinish','Fence','PoolQC']

ordinal_orders=[ 
    #LotShape 
['Reg','IR1' ,'IR2','IR3'], 
    #ExterQual 
['Fa','TA','Gd','Ex'], 
    #ExterCond
['Po','Fa','TA','Gd','Ex'],
    #BsmtQual 
['None','Fa','TA','Gd','Ex'], 
    #BsmtCond 
['None','Po','Fa','TA','Gd','Ex'], 
    #BsmtExposure 
['None','No','Mn','Av','Gd'], 
    #BsmtFinType1 
['None','Unf','LwQ', 'Rec','BLQ','ALQ' , 'GLQ' ],
    #BsmtFinType2 
['None','Unf','LwQ', 'Rec','BLQ','ALQ' , 'GLQ' ], 
    #HeatingQC 
['Po','Fa','TA','Gd','Ex'], 
    #KitchenQual 
['Fa','TA','Gd','Ex'],
    #FireplaceQu 
['None','Po','Fa','TA','Gd','Ex'], 
    #GarageQual 
['None','Po','Fa','TA','Gd','Ex'], 
    #GarageCond 
['None','Po','Fa','TA','Gd','Ex'], 
    #GarageFinish 
['None','Unf','RFn','Fin'], 
    #Fence 
['None','MnWw','GdWo','MnPrv','GdPrv'], 
    #PoolQC
['None','Fa','Gd','Ex']
]

transformer = make_column_transformer((OrdinalEncoder(categories= ordinal_orders),ordinal_features), remainder = "passthrough").fit(X_train)

Then you can apply the transformer to both X_train and X_test

transformer.transform(X_train) or transformer.transform(X_test)

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you!! Yes, the reason for the convoluted method was because i kept getting errors when trying to apply it simply, errors such as shape mismatch and a few others. I did manage to make it work eventually by messing around with the number of square brackets around the features when putting them into the encoder. How would you recommend i apply this straight to my train and test dataframe? $\endgroup$
    – Zèro
    Jan 11, 2021 at 19:37
  • $\begingroup$ I would always recommend using Pipelines to avoid any data leakage, In that way you can pass the transformer to a Pipeline and It will always be the cleanest and correct way $\endgroup$
    – Multivac
    Jan 11, 2021 at 19:40
  • $\begingroup$ You can apply the transformer to whichever set you need, either train or test set, the important part is that the transformer is fitted only on the train set $\endgroup$
    – Multivac
    Jan 11, 2021 at 19:50
0
$\begingroup$

In the documentation categories parameter is explained as: categories[i] holds the categories expected in the ith column. The passed categories should not mix strings and numeric values, and should be sorted in case of numeric values.

You can pass a two dimensional array to the categories parameter in which each element of the array is an another array that holds categories of corresponding column. In the case of numeric values, it is stated that it should be sorted. So maybe it can work:

cats = [list(df[c].sort_values().unique()) for c in df.columns]
encoder = OrdinalEncoder(categories=cats)
Improve this answer
$\endgroup$
1
  • $\begingroup$ I tried a few things with your approach but i ended up in a position where i would essentially have to apply a new .fit.transform as my 'cats' would be the new data from the initial .fit.transform. Meaning i'm no longer applying a transform. Thanks anyway! $\endgroup$
    – Zèro
    Jan 11, 2021 at 16:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.