I'm new to data science and stats, so this might seems like a beginner question.
I'm working on a dataset where I've user's Twitter followers gain per day. I want to measure the average growth he had over a period of time, which I did by finding the mean of growth. But someone is suggesting me to use median for this.
Can anyone explains, in which use-case we should use mean and when to use median?
The arithmetic mean is denoted as $\bar{x}$
$$\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i $$
where each $x_i$ represent an unique observation. The arithmetic mean measures the average value for a given set of numbers.
In contrast to this, the median is the value which falls directly in the middle of your dataset. The median is especially useful when you are dealing with a wide range or when there is an outlier (a very high or low number compared to the rest) which would skew the mean.
For example, salaries are usually discussed using medians. This due to the large disparity between the majority of people and a very few people with a lot of money (with the few people with a lot of money being the outliers). Thus, looking at the 50% percentile individual will give a more representative value than the mean in this circumstance.
Alternatively, grades are usually described using the mean (average) because most students should be near the average and few will be far below or far above.
It depends what question you are trying to answer. You are looking at the rate of change of a time series, and it sounds like you are trying to show how that changed over time. The mean gives the reader one intuitive insight: they can trivially estimate the number of followers at any date $d$ days since the start by multiplying by the mean rate of change.
The downside to this single metric is that it doesn't illustrate something which is very common in series such as this: the rate of change is not fixed over time. One reasonable metric for giving readers an idea of whether the rate of change is static is giving them the median. If they know the minimum of the series (presumably zero in your case), the current value, the mean and the median, they can in many cases get a "feel for" how close to linear the increase has been.
There is a great cautionary tale in Anscombe's quartet - four completely different time series which all share several important statistical measures. Basically it always comes back to what you are trying to answer. Are you trying to find users which are likely to become prominent soon? Users which are steadily accruing followers year by year? One hit wonders? Botnets?
As you've probably guessed, this means it's not possible to universally call mean or median "better" than the other.
Simply to say, If your data is corrupted with noise or say erroneous no.of twitter followers as in your case, Taking mean as a metric could be detrimental as the model will perform badly. In this case, If you take the median of the values, It will take care of outliers in the data. Hope it helps
I find myself explaining this a lot and the example I use is the famous Bill Gates version. Bill Gates is in your data science class. Your instructor asks you: what is the average income or net worth of this class? Bill Gates sheepishly obliges and tells you what his income is. Now when you say the average income of your group is a zillion dollars - technically correct but does not describe the reality - that Bill Gates is an outlier skewing everything.
So you line up all the people in your group in ascending or descending order - whatever the person in the middle is making - that is your median. In this example, everybody but Bill Gates is likely to be in spitting distance of that median, and Bill Gates will be the only one making anything close to the mean.
Now say buddy Bill Gates is hiring a money manager. Based on the returns they produced so far. Should he look at their average returns over a 10 year period or their median return or a combination of the two? Did they outperform the market each year? Some years? How does portfolio size factor in? In the case of Twitter followers, Obama would have a different growth compared to someone with say 500K-1MM followers. As @l0b0 alludes to in his excellent answer - it all depends. Are you measuring follower growth or the rate of change of follower growth and what is the question you are trying to answer, strategy/product you are trying to develop - accordingly you pick mean or median. Getting the mean and median is always the easy part. It's always better to never ever have the average of 2.1 kids. Have a whole number of kids. But what can you say about population growth rates if mean number of kids is 2.1 and median is 1 or 2? Or median is 3 or more? Is growth accelerating or decelerating? What is mode doing? Compute all the basics first - and then ask the reason why you are using mean versus median.
Often median is more robust to extreme value to mean. Try to think it as a minimization task. Median corresponds to absolute loss while mean corresponds to square loss.
I was trying to find a center where the absolute difference with all the other values are minimized. It turned out to be median.
$$ f(center) = \sum | center - x | $$
After we sort all the values, we can make a graph with the sum of absolute difference in Y axis and the center values in the X axis. The graph will be a inverse bell shape. And the bottom of the curve is where center=median .
It is hard to get it from intuition.
