Correlating & Combining categorical and numerical features for classification problem

Question

I just started with my first Machine Learning project (Jupyter Notebook, Python, Scikit-learn, pandas) and I am working on a Palmer Penguin Dataset. I have been doing some data analysis and I got stuck when trying to correlate and combine some of the attributes from the dataset.

1. Correlation.

The dataset contains 2 categorical (gender, island) and 4 numerical (body weight, flipper length, culmen depth, culmen width) features as well as a label (penguin species). I was first trying to correlate the features with the label - I have used LabelEncoder from scikit-learn to transform the species label into a numerical attribute since the correlation function of pandas dataframe only works on numerical attributes. Even though I got some results, I have read from many sources that it is an improper approach for measuring the correlation between categorical and numerical attributes. I would appreciate it if somebody can clarify for me, what is the preferred way of measuring the correlation of such types.

2. Combining.

One of the reasons why I wanted to measure a correlation between those attributes is to possibly replace some of the attributes with combined attributes if it would make sense. One of the combination that is logically making sense is combining a gender with a body weight - I was thinking that gender separately does not bring much value, but it could be more valuable if we combine it with the body weight. Does such a combinations make sense and can they improve the accuracy of the classification model? If yes, how to approach combining categorical with numerical attributes using specified tools?

Answer 1

Regarding your first point, you can correlate your features to your target the way you have done (to my knowledge. I'm not 100% sure). This is because you are only converting the categorical classes into numerical so in the end you might get results like feature 1 has a correlation with class 1 of about 0.4 and so on. Then you can reverse your LabelEncoder encoding to get back the categorical classes i.e class 1 is species A and class 2 is species B. I think it should be correct but someone please correct me if I am wrong.

Regarding point 2, what you describe is known as feature engineering, where you combine multiple features into a single feature based on domain knowledge. If you do not have domain expertise then I would suggest to stay away from combining features.

The example you gave does not make sense to me as gender is a categorical feature and weight is a numerical feature. So how can you combine both of them? It will probably lead to incorrect/bad results.