0
$\begingroup$

Doing simple supervised regression where the label is a floating point number (guaranteed positive) and the features are a mix of continuous floating point values and some categorical features.

What would be a good metric of global bivariate dependence that would assign a value to each feature, based on the training dataset. By that I am trying to say

  • The importance will be calculated for each feature, not each feature and each sample
  • Best if all of them are positive and add up to one, which can be easily achieved by doing a SoftMax as the final step after getting the values otherwise.
  • The importance should not be dictated by any specific model, such as CatBoost or XGBoost etc. Just a measure based on the data alone.

If all the features were continuous floats, I would just take the Pearson correlation, but some are categorical as I pointed out. Even for categorical, likely measures like mutual information or entropy can help, but then it is difficult to compare the numbers among features.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I think you may get better answers if you ask for suitable measures of bivariate dependence, rather than feature importances. "Feature importance" is usually associated with a particular model, so the request for model-independent feature importance seems contradictory and might not yield the best answers. There's a list of measure that might be of interest here: 1library.net/article/… $\endgroup$
    – KishKash
    Commented Nov 16, 2022 at 9:04
  • $\begingroup$ @KishKash thanks for the comment, and correcting the terminology. Yes, that is what I want and I edited the questions. $\endgroup$
    – Della
    Commented Nov 16, 2022 at 9:25

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.