0
$\begingroup$

I would have a question on the contingency table and its results. I was performing this analysis on names starting with symbols as a possible feature, getting the following values:

Label          0.0  1.0     
with_symb      1584 241
without_symb     16 14

getting a p-value which lets met conclude that variables are associated (since it is less than 0.05). My question is if this result might be a good result based on the chi-squared test, so if I can include in the model. I am selecting individually features to enter the model based on the chi-squared. Maybe there is another way to select the most appropriate and significant features for the model. Any suggestions on this would be great.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I will raise several issues that could arise if you are selecting features based on chi-2 tests

  1. Repeated use of chi-2 test can lead to spurious results unless you correct for the number of times you run it

  2. You can include features that are correlated with each other, i.a. A is correlated with B, and both are correlated with label. Not sure, but I think, this can lead to results where model performs worse with more features.

I would try starting with all the features, remove the ones linearly correlated. But this is just a suggestion.

Also, mutual information can be used to estimate how well any given feature describes the label.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thanks a lot, Cyro. Yes, I think it makes sense your suggestion. May I ask you how to consider all the features and remove those ones which are correlated? I am using python, but even a small suggestion on what to look at in terms of statistical test to achieve this, it would be great! $\endgroup$
    – V_sqrt
    Feb 8, 2021 at 16:51
  • 1
    $\begingroup$ I have seen people use Variance Inflation Factor. Personally, I would build the correlation matrix (i.e. covariance matrix with standard deviations normalized to 1), get the eigenvector that corresponds to lowest eigenvalue, and remove the feature that has largest projection along that feature. Then repeat the process, and carry on going until smallest variance is above the value you deem acceptable. Having said that it may be a good idea to look at the correlation matrix spectrum before that to see whether there are any obvious interesting patterns. $\endgroup$
    – Cryo
    Feb 8, 2021 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.