I know this question has been asked before and I have tried a few things but those things are not working as expected for my usecase.
I have a 500 length feature vector. One of these features is a categorical value pincode. For our dataset, the pincode can take more than 20,000 unique values. So we can't used one hot encoding because it will blow up our feature space.
I have also tried Binary Encoding which assigns a unique integer to every unique categorical value and then converts it into Binary. It then treats every bit as a column. This way the dimension of the pincode feature vector had reduced from 20,000 to around 20. But I didn't find any improvement in our evaluation metrics which is AUC (Area under Curve). Intuitively also it didn't felt right thing to do.
I also tried to apply PCA on the pincode to reduce the dimensions. I tried to reduce the dimensions from 20,000 to 100. Upon applying PCA, the model performed worse. Also, I read somewhere that PCA doesn't work well on categorical values. It's better for continous values.
So what do we do with this feature? We don't want to throw it away as we think that it might be an important feature. But we want to reduce it dimension and then use.
Apologies if this is a basis question. I am new to this field and trying various things out.
P.S - We are using xgboost for training our model.