Working on the World Happiness Report dataset, i have N countries with M features and a happiness score. This is the parameter I built 3 classes from: happy, medium, unhappy (numerical intervals of happiness score). I have run a 3 component PCA (only the first two components show relevant separation) and kmeans on the result (with 6 clusters, chosen in function of distorion). Plotting the two components with clustering predicted colouring gives very clear separation between classes, and I should now deduce what feature makes "happy" countries happy. My idea would be to implement some form of Bayes' theorem, but don't really know how since there is strong (>0.6) correlation between some features. How can I approach the problem?
If they are highly correlated, probably you can not easily tell which feature leads to a happy country. My suggestion is to perform multicollinearity test before fitting any model to remove highly correlated features. After that, there a chance that you be able to get more insights about the pattern in your data.