6

No. What you refer to (the difference between the means of group A and B) is actually the effect size, and it has absolutely nothing to do with the p-values. The situation is nicely summarized in the (highly recommended) paper Using Effect Size—or Why the P Value Is Not Enough (emphasis mine): Why Report Effect Sizes? The effect size is the main finding of ...


2

According to The Search for Categorical Correlation post on TowardsDataScience, one can use a variation of correlation called Cramer's association. Going categorical What we need is something that will look like correlation, but will work with categorical values — or more formally, we’re looking for a measure of association between two categorical features. ...


1

can you apply a trained algorithm / model on its own training data if my interest is not in predicting forward but evaluating success vs likelihood of success (and still yield useful information)? Yes, absolutely. There are quite a few cases where applying a model on the training data is useful. The most common is probably to detect overfitting: a high ...


1

The notation $D_\mathrm{KL}(P \| Q)$ is pretty much standard for the Kullback-Leibler divergence. There is an interesting discussion on Mathematics SE on the reasons for the fairly unusual notation used for divergences. In general, $x \sim G(z, c)$ means "$x$ is a random variable distributed according to $G(z, c)$". In the subscript the author ...


1

In hierarchical clustering, both agglomerative and divisive, you do not have to pre-specify the number of clusters. You can create all possible clusters and then select the number cluster to use at the end.


1

I'm going to just answer about the difference between the two functions qcut and cut because it's a very important difference: the first qcut is indeed about quantiles, which means that it's about dividing the data into bins, each containing an equal number of points. For instance if you use deciles it means that there is the same number of people in the ...


1

There's no way to have a complete summary of a large dataset like this, you have to analyze what can be relevant, decompose into more specific pieces of information and then find the best way to visualize each specific part on its own. The first thing would be to plot the distribution of this parameter of interest across subjects and/or observations. If you ...


1

To start with, you could use a simple thresholding. If you have the dataset $S$ where an element has the form $(x,y,c) \in S$, $x$ denotes the year, $y$ is a binary value (exam passed or not), and $c$ is the student id. you can obtain a classifier by using $\{(x,y,c) \in S \mid x \leq \theta\}$ and $\{(x,y,c) \in S \mid x > \theta\}$. Now you can check ...


1

Most probably, you are using Pearson's correlation method. This method is used for two Continuous features. Here, both the price_drop and the OHE features are Binary Categorical features. So, you can use these methods - Phi - Phi is a measure of the degree of association between two binary variables (two categorical variables, each of which can have ...


1

This information is in the documentation of Seaborn. They show a bootstrap confidence interval, computed by resampling units (rows in the 2d array input form). By default, in seaborn version 0.8.1 it uses 95% of confidence interval, which is equivalent to a standard error. This value is a parameter that can be changed.


Only top voted, non community-wiki answers of a minimum length are eligible