7
$\begingroup$

I have following business domain. I have a product with three outputs/labels. The outputs are impacted by 1000 procedures, each procedure is digitized and measured. The customer wants to know what is the most influential procedures on the outputs.

1. From Pearson correlation coefficient we could learn how two variables' relationship, say 1 is proportional, -1 is negative proportional and 0 is no relation. So I could find the biggest value of Pearson correlation coefficient to find more influential procedures.

2. From Random Forest algorithm, I could know the top feature importance. So I could identify also the most influential procedures.

Which one is better?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
10
$\begingroup$

Pearson correlations capture linear relationships between the input and target variables. Therefore this only makes sense for continuous inputs and a continuous target variable, and not continuous inputs with a binary/categorical output. Correlations essentially measure the positive/negative 'change' in one feature as you increase/decrease the other.

So it doesn't make much sense to compare the relationship between your input features and the categorical outputs this way. You may as well calculate the mean input for each feature and each label, and calculate the differences between those. I found this answer on Cross-Validated which explains this much better than I can.

Feature importance in tree based models is more likely to actually identify which features are most influential when differentiating your classes, provided that the model performs well. How this feature importance is calculated depends on the implementation, this article gives a good overview of how different tree based models calculate importance for features.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ This beautiful picture is for continuous-continuous variables. Continuous-categorical (feature-label) case is different, since "linear" relation has no meaning. $\endgroup$
    – Esmailian
    Commented Mar 21, 2019 at 14:43
  • 1
    $\begingroup$ Ah well noticed, I hadn't spotted this question was asking about categorical labels, I'll edit my answer :) $\endgroup$
    – Dee Carter
    Commented Mar 21, 2019 at 15:12
0
$\begingroup$

I would say it depends a bit on what you want to achieve.

A few things to keep in mind:

Pearson gives you a correlation but what is if the information is in the absolute value- a RF has a much better chance to recognize this. Example data where there is some clear correlation but in the absolute value:

a = [1,1,1,0,0,0, -1,-1,-1]
b = [abs(x) for x in a]

On the other hand RF importance is only relevant when the prediction is good - whatever good means for you. Pearson R has a very specific meaning that is always true- there is a correlation between the two variables.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.