The dataset I have is quite small (130 samples, 6 features). I am interested in finding relationships between the features.

For example, I found a linear relationship between feature1 and feature2. I did this by calculating Pearson's correlation for each pair of features (In this case there will n*(n-1)/2! = 15 where n is the number of features). From this I found that the correlation between feature1 and feature2 was large (some other pairs as well).

But now suppose I am interested in finding, say, a parabolic relationship between two variables. I don't know if there is something analogous to Pearson's correlation. In any case, one way to proceed would be to simply perform quadratic regression for each pair of features and see if any of these models perform well. Is there a better approach than this? Another approach would be to look at the graphs of all feature pairs and see if any look parabolic, but for more features this becomes too time consuming.

  • 1
    $\begingroup$ As you have pointed out, correlation only describes linear relationships. Depending on your goals, you could try different ways of describing relationships -- for instance, if you need less interpretable relationships, you could try fitting any model (XGBoost for instance) and getting feature_importances_ from that model -- it will return the average gain by the feature. or in LightGBM it will be the number of times the feature is used in gradient-boosted decision trees. $\endgroup$
    – Niqua
    Commented May 15 at 12:21
  • $\begingroup$ If you need more interpretable ways of describing relationships, you could even try SHAP: shap.readthedocs.io/en/latest $\endgroup$
    – Niqua
    Commented May 15 at 12:22


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.