Collinearity and Outlier Removal

Question

I am playing with a credit fraud detection dataset at Kaggle. An imbalanced dataset with about 0.1% of fraud transaction. The features are 28 PCs out from a PCA exercise done by the data publisher + time & txn amount and a class variable of 0/1 for legit/fraud txn.

From my brief understanding, collinearity should have been dealt with during PCA. However, I found that PCs are still correlated among the fraud cases (if you break the dataset up into legit/fraud cases). What should be a good approach to minimise that effect for fraud detection using a Naive Bayes classifier?

Another thing is that I have been taught in DS101 to deal with outliers. However, I don't seem to think removing outliers is a wise choice given that fraud can be an outlier by itself. What are some common approach to deal with outliers while not removing them?

P.S. I am fairly new to Data Science so any good directions on the above topics would be welcome. It is just not as clear cut as I have seen in the introductory text.

Thanks.

Answer 1

In general machine learning algorithms, if feeded with large training datasets are able to deal both with outliers and multicollinearity. PCA is a dimension reduction techniques and surely helps with multicollinearity. Naïve Bayes assumes independence of its input features (the word naïve comes from this property). So after PCA Naïve Bayes has more chance to get better results. If you find that PCs are still correlated among the fraud cases I don't think it is an issue. However, you could try to preprocessing your data removing highly correlated variables according to some criteria. The caret library has many functionalities for preprocessing and this tutorial cover interesting stuff besides software applications.

https://topepo.github.io/caret/pre-processing.html

About outliers I am always skeptic about any outlier removal except in the case could be due to miscoding.