16 votes
Accepted

How can I ensure anonymity with queries to small datasets?

This is a difficult problem, regardless of how you look at it. But some solutions might be good enough. Here, you have the advantage that you are returning aggregate statistics, and not individual ...
amon's user avatar
  • 276
15 votes

When to use mean vs median

It depends what question you are trying to answer. You are looking at the rate of change of a time series, and it sounds like you are trying to show how that changed over time. The mean gives the ...
l0b0's user avatar
  • 251
12 votes
Accepted

When to use mean vs median

The arithmetic mean is denoted as $\bar{x}$ $$\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i $$ where each $x_i$ represent an unique observation. The arithmetic mean measures the average value for a given ...
JahKnows's user avatar
  • 8,856
7 votes
Accepted

p-value and effect size

No. What you refer to (the difference between the means of group A and B) is actually the effect size, and it has absolutely nothing to do with the p-values. The situation is nicely summarized in the (...
desertnaut's user avatar
  • 1,958
7 votes

How can I ensure anonymity with queries to small datasets?

As the other answer points out, total anonymity is difficult to accomplish especially when considering an adversary who is trying to break your data privacy scheme. But, that might not be what you're ...
David's user avatar
  • 171
5 votes
Accepted

Range to define emotions

The final range of emotion is completely arbitrary. No matter the interval [a, b], you can adjust the emotions to fit inside. [-100, 100] is perfectly reasonable and is common. An example of use is ...
Stefan G.'s user avatar
  • 206
4 votes

When to use mean vs median

Simply to say, If your data is corrupted with noise or say erroneous no.of twitter followers as in your case, Taking mean as a metric could be detrimental as the model will perform badly. In this case,...
karthikeyan mg's user avatar
3 votes

When to use mean vs median

I find myself explaining this a lot and the example I use is the famous Bill Gates version. Bill Gates is in your data science class. Your instructor asks you: what is the average income or net worth ...
armipunk's user avatar
  • 131
3 votes
Accepted

What do you call the ratio of positive to negative samples?

You can use class ratio / sample class ratio. Which will make it more intuitive for any reader while going through the details. As its not used for model performance analysis hence I think we don’t ...
cap's user avatar
  • 432
3 votes

Distribution vs Histogram

You could think of histogram as one way of plotting a distribution of values. Another way of plotting such distribution is a KDE (Kernel Density Estimate), but after all, these inform about the same ...
German C M's user avatar
  • 2,696
2 votes

Difference between Blocking and Clustering?

As you mentioned their goals are different. In clustering, we try to group data such that they have the same variability. For example, clustering customers of a company into different clusters, ...
OmG's user avatar
  • 1,219
2 votes

How to measure the performance of an imputation technique

If you just want to do descriptive analysis it might be a good idea to do this without imputation at all. A more complicated method is the following: You take the data without missing values from ...
Steffen Moritz's user avatar
2 votes

How to measure the performance of an imputation technique

I think there is no answer to your question since there is no absolute universal "good". Everything depends on the question you ask and the tools you use. This is why there are a lot of imputation ...
rapaio's user avatar
  • 4,733
2 votes
Accepted

Normal distribution and QQ plot

The normal distribution is a theoretical model of data. Empirical data can be distributed more similarily or more dissimilarly to a normal distribution. That empirical data has a couple of notable ...
Brian Spiering's user avatar
2 votes

Skew and Kurtosis are so similar?

A simple and direct answer is that skewness and kurtosis are both defined in terms of the $Z-$values, $Z = (X-\mu)/\sigma$ : Skewness = $E(Z^3)$ and Kurtosis = $E(Z^4)$. When talking about a data set, ...
BigBendRegion's user avatar
2 votes

Approach to creating a user profile in music web application

You can download a free as in beer software Qlikview that allows you to do interactive data discovery via graphical interface similar to Excel but also featuring a powerful scripting language for data ...
Diego's user avatar
  • 550
2 votes

When to use mean vs median

Often median is more robust to extreme value to mean. Try to think it as a minimization task. Median corresponds to absolute loss while mean corresponds to square loss.
nan hu's user avatar
  • 21
2 votes

Generating random numbers from best probability distributions?

Good question. Lots of options. Most would recommend you use CDFs instead of histograms. So convert your observed distribution to empirical CDF (in R it’s ecdf() - I dint know python). Then, plot ...
HEITZ's user avatar
  • 911
2 votes

Which statistical test tells which classifier performs better than the other?

Cochran's Q test Is a generalisation of the McNemars test and can be used to see if there is a truly better classifier for the metric chosen. You can ofcourse also do pairwise Mcnemare test and draw ...
Noah Weber's user avatar
  • 5,669
2 votes

How to interpret axis of Histogram and distribution curve?

First figure shows Frequency(Y-axis) distribution over varied values of Line data(X-axis). Similar information gets conveyed by your second figure as well, but second one provides a deeper insight to ...
Random Nerd's user avatar
2 votes
Accepted

Problem regarding calculating correlation approach?

Most probably, you are using Pearson's correlation method. This method is used for two Continuous features. Here, both the price_drop and the OHE features are Binary Categorical features. So, you can ...
10xAI's user avatar
  • 5,574
2 votes
Accepted

Calculating correlation for categorical variables

According to The Search for Categorical Correlation post on TowardsDataScience, one can use a variation of correlation called Cramer's association. Going categorical What we need is something that ...
Nikos M.'s user avatar
  • 2,301
1 vote

Aggregate several attributes into one using PCA

PCA will change your data and you will not be able to interpret it in sane sense, you can just slice and dice the data and do many things by hand, PCA would be usefull if you would want to find "...
quester's user avatar
  • 295
1 vote

Aggregate several attributes into one using PCA

I see. Yes PCA is a good tool that you can use on your choice of attributes. The tricky part is that PCA is based on variance. Hence, if you have 2 attributes which are more important and have low ...
20-roso's user avatar
  • 680
1 vote
Accepted

Looking for help calculating a probability formula

Ok, I am not sure if I understood it correctly. But if it is only the right-side figure equation it would be: =1/(1+EXP(3.1058)) (I have tested on LibreOffice ...
Victor Oliveira's user avatar
1 vote

When to use mean vs median

I was trying to find a center where the absolute difference with all the other values are minimized. It turned out to be median. $$ f(center) = \sum | center - x | $$ After we sort all the values, ...
KRoy's user avatar
  • 111
1 vote

How does Seaborn calculate error bars when using estimators other than the arithmetic mean?

This information is in the documentation of Seaborn. They show a bootstrap confidence interval, computed by resampling units (rows in the 2d array input form). By default, in seaborn version 0.8.1 it ...
Carlos Mougan's user avatar
1 vote

What is the first tool to learn start your your data science projects?

This question was probably asked million times. Please try to find answers berfore you ask "where to learn start".
Michał Kardach's user avatar
1 vote

How do I handle string feature while performing model generation

What you need to do is called One Hot Encoding. There are two ways to do. One is using Scikit-learn as described in Scikit-Learn documentation or use get_dummies ...
Tasos's user avatar
  • 3,910
1 vote

Binary Classification without machine learning tools

How about implementing one of the most basic binary classifiers by hand without any libraries, Logistic Regression? Here is the explanation of the Logistic Regression in subtle terms: https://www....
Ugur MULUK's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible