For standard deviation's formula, why does division by sample size come before square rooting?

Question

Why is this used to calculate a sample's standard deviation:

$s=\sqrt{\frac{1}{N-1}\sum_{i=1}^N(x_i-\bar{x})^2}$

and not something like:

$s=\frac{1}{N-1}\sqrt{\sum_{i=1}^N(x_i-\bar{x})^2}$

I understand the purpose of squaring then square rooting is to regard only the distance (which can only be positive) between the two points. If thats the case why not just do it directly before getting the sample size involved?

Answer 1

Standard deviation is taken to be the square root of the variance, and the variance divides by the sample size. If you want to define some other statistic that takes the square root and then divides by the sample size, that’s fine, but standard deviation and the variance to which it is so closely related arise quite naturally in the probability theory on which much of statistics is based. A few that come to mind:

1. Standard deviation plays a unique role in the central limit theorem

2. Standard deviation comes up in the 68-95-99.7 “empirical rule” for normal distributions and its generalization through the Chebyshev inequality

Relationships between standard deviation and standard errors in statistical inference (e.g., confidence intervals, t-testing) could be another reason to care about the standard deviation (though your formula is quite close to the standard error used for basic confidence intervals).

Being really formal, we use $S=\sqrt{ \frac{1}{n-1}\sum(x_i-\bar x) }$ as an estimator of population standard deviation, and this estimator tends to be close to the true value. If you use your equation to estimate the standard deviation, it will tend to be too small. This can be demonstrated in a simulation. I will do one with a population standard deviation of $1$.

library(ggplot2)
set.seed(2023)
N <- 5
R <- 1000
sd_usual <- sd_you <- rep(NA, R)
for (i in 1:R){
    
    x <- rnorm(N, 0, 1)
    sd_usual[i] <- sd(x)
    sd_you[i] <- sum((x - mean(x))^2)/(N - 1)


}

d0 <- data.frame(
    estimate = sd_usual,
    estimator = "Usual"
)
d1 <- data.frame(
    estimate = sd_you,
    estimator = "Yours"
)
d <- rbind(d0, d1)
ggplot(d, aes(x = estimate, fill = estimator)) +
    geom_density(alpha = 0.25) +
    geom_vline(xintercept = 1)

Which estimator seems to do a better job of estimating the true value of $1?$ I say it’s the usual one in pink!

Answer 2

Firstly, total variance estimate is divided by the degree of freedom (N - 1) to compute/estimate the "average" variance for the sample. Following this, we can estimate the standard deviation i.e. square root of average variance estimated. But, your proposal will produce s i.e S.D. equal to square root of sum total of variance of data divided by degree of freedom. It produces simple average variance. It is not supported by any statistical theorem/Sampling theory.