0
$\begingroup$

I know this question has been asked before and I have tried a few things but those things are not working as expected for my usecase.

I have a 500 length feature vector. One of these features is a categorical value pincode. For our dataset, the pincode can take more than 20,000 unique values. So we can't used one hot encoding because it will blow up our feature space.

I have also tried Binary Encoding which assigns a unique integer to every unique categorical value and then converts it into Binary. It then treats every bit as a column. This way the dimension of the pincode feature vector had reduced from 20,000 to around 20. But I didn't find any improvement in our evaluation metrics which is AUC (Area under Curve). Intuitively also it didn't felt right thing to do.

I also tried to apply PCA on the pincode to reduce the dimensions. I tried to reduce the dimensions from 20,000 to 100. Upon applying PCA, the model performed worse. Also, I read somewhere that PCA doesn't work well on categorical values. It's better for continous values.

So what do we do with this feature? We don't want to throw it away as we think that it might be an important feature. But we want to reduce it dimension and then use.

Apologies if this is a basis question. I am new to this field and trying various things out.

P.S - We are using xgboost for training our model.

$\endgroup$
  • $\begingroup$ What kind of algorithm are you using? $\endgroup$ – Dan Scally Sep 20 at 9:11
  • $\begingroup$ we are using xgboost for training. $\endgroup$ – Anshul Jain Sep 20 at 9:13
0
$\begingroup$

one idea is to craft some features of this feature - when you classify cars you don't have data like strings "Ferrari 991 year 2014 red" "BMW z4 year 1999 2.0L blue" but you would like to have columns like "Manufacturer", "Type(SUV/Cabrio...)", "Year", "Engine" etc.

you could transform it or just discard

$\endgroup$
0
$\begingroup$

Pincode I feel very very likely is not a predictor at all for the target variable you have. If you feel geography is an important predictor and has bearing on you target variable then what you can do is use city or state or other relevant geographical unit like sub-city etc. you feel have homogeneous characterstics.

Say if state X n Y each has 2 cities P,Q and M,N and ecah city is very different from other one then I will choose city as my geographical variable otherwise state as Geo variable is fine. So it depends on homogeneity of Geo unit.

For example if say you are predicting demand for luxury cars and city A has great roads, high income population etc. and thus high demand for luxury cars compared to city B which has none of features city A has. So in your model city A observations have more probability of buying a luxury car then city B observations.

So by using a bigger Geo unit than pincode you end up reducing categories say from 20000 pin codes you can have only 100 cities which is easily managable by xgb.

So use a bigger Geo unit if you feel geography affects your target variable.

Best Vik

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.