I'm stuck while working on a categorical data analysis project and would appreciate any insight.
Here are the first 4 rows of the dataset:
| successful_upload | feat_1 | feat_2 |
|---|---|---|
| 0 | T>2 | <\$10K |
| 1 | 0.5<T<2 | >\$10K |
| 0 | T<0.5 | losing |
| 0 | T>2 | equilibrium point |
| ... | ... | ... |
In my current dataset there is a category (>10k) that seems to have a significant impact on the target variable (named success_uploading) when visually inspected.
Here is the plot. T represents the age of the company. As seen, earning >10K influences the target variable in great deal.
Looking more info about this, found a research paper (https://doi.org/10.3390/math9070746), which employs the $\chi^{2}$ (chi-squared) test to identify features with higher predictive power for the target variable.
From the table it can be seen that both features and the target variable are categorical. And before doing any statistical analysis, I used one-hot encoding method.
Then, to quantify the association, I applied the $\chi^{2}$ test, as stated in the paper. However, despite the promising visual patterns, in the application of the $\chi^{2}$ test the p-value associated between one-hot-encoded variable >$10K and successful_upload did not allow me to reject the null hypothesis, which means that the variables are independent.
This discrepancy between the visually evident impact of >$10K on successful_upload and the independency between them is not explainable for me.
Has anyone encountered a similar situation before? How should I interpret this scenario? Are there other techniques or statistical tests that I should consider? or perhaps I am making a mistake. :(
Thank you in advance for your help.
