1
$\begingroup$

I have a dataset containing insurance Claims with quantitative and qualitative variables but PCA refuses to convert or work with "string" type variables.

This is the code I used :

from sklearn.decomposition import PCA
claims=pd.read_csv('./insurance_claims.csv',sep=',',header=0)
X=claims.ix[:,1:].values
pca=PCA(n_components=12)
pca.fit(X)

I'm trying to reduce dimensionality to cluster the dataset and detect fraudulous claims. If there is any alternative to PCA for heterogenous data much appreciated.

$\endgroup$
1
$\begingroup$

Yes, Since machines cannot quantify qualitative (or) sometimes called categorical data. We manually have to quantify data when processed along with quantitative data. One way of converting these categorical data is One hot encoding, By performing one hot encoding, you convert categorical data into numerical data, something that a machine can understand There are some cases where one hot encoding fails. When the number of categories are high, you will get highly sparse features which makes no sense Ex: Countries as categorical labels. In these types of situations you can calculate distance of all countries from a certain point on the earth Ex: Calculating the distance of each country from equator. I'm just a beginner in DS.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Hi, OK I will try One Hot Encoding using Sklearn or other libraries. Thanks! $\endgroup$ – Soufiane Sabiri Jan 23 '19 at 11:56
  • $\begingroup$ Note that when you do PCA, your variables do not retain the original meaning they had before the transformation. Thus, if you're looking to retain interpretability, you might need to try another method. $\endgroup$ – Victor Ng Oct 21 '19 at 13:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.