Let's say there is a distribution, call it D, for which I don't know details (i.e. mean and variance) but can assume that it's a normal distribution. I now have N samples from D. I cannot take more samples because it's too heavy. Given the N samples, and the assumption that D is a normal distribution, how can I compute the mean and variance of the mean of D? Naturally it's less likely for the mean of D to be far away from the samples (thought still possible), while more likely to be near the samples.
The same can be done through any of the statistical estimation technique such as maximum likelihood or Minimum variance unbiased estimator(MVUE). Generally maximum likelihood is the easiest way to evaluate the mean and variance of the distribution. If you perform Maximum likelihood method the estimated mean of the normal distribution is the sample mean. Given x1,x2.....xn are the samples
Estimated mean u = (x1 + x2 + ... + xn)/n Similarly the estimated variance is Σ(xi - u)^2/n
Both of these are obtained by setting the partial derivative of the likelihood function with respect to the mean and the standard deviation to zero.
Hope this helps.