I am new to Data Science. Recently I was studying a course about statistics. One of the tasks there was to check the central limit theorem in practice.

The idea was quite simple: take a continuous random variable; generate, say, 1000 samples from it, each of size n; draw a histogram of the samples. Then find the parameters of the normal distribution using the central limit theorem and draw the PDF of the distribution. As a result, the histogram and the PDF should be, roughly speaking, "similar" (and become more "similar" as n grows).

I chose Pareto distribution and, with this Python code,

import numpy as np
import matplotlib.pyplot as plt
import scipy.stats as sts
from math import sqrt

n = 10

b = 3.0
random_value = sts.pareto(b)
mean = random_value.mean()
variance = random_value.var()

plt.hist(samples(random_value, n, 1000), bins=20, normed=True)

x = np.linspace(0, 6, 100)
pdf = sts.norm(mean, sqrt(variance / n)).pdf(x)
plt.plot(x, pdf, color='r', label='theoretical PDF')

where the samples function was like this

def samples(random_value, sample_size, number_of_samples):
    result = np.asarray([])
    for i in range(number_of_samples):
        result = np.append(result, np.mean(random_value.rvs(sample_size)))
    return result

I got the next histogram and PDF as a result Pareto distribution

The result looked suspicious to me: there was no expected similarity. I checked the same code with another continuous distribution (with a uniform distribution), and the graphs looked much more similar.

Is it a feature of the Pareto distribution which makes it a bad choice to demonstrate how the central limit theorem works? Or something is wrong with the code itself (e.g., the parameters of the normal distribution are calculated in a wrong way)? Thank you in advance.

  • 3
    $\begingroup$ I'm voting to close this question as off-topic because there's better statistical expertise at crossvalidated: stats.stackexchange.com $\endgroup$
    – Spacedman
    Jun 17 '18 at 22:02
  • 1
    $\begingroup$ where does random_value.rvs come from? Please try and make a reproducible example. $\endgroup$
    – Spacedman
    Jun 17 '18 at 22:10
  • $\begingroup$ @Spacedman, thank you for the hint. I have updated the code, now it is exactly the same I am executing. $\endgroup$
    – yaskovdev
    Jun 18 '18 at 9:34

One key assumption of the central limit theorem is that the variance $\sigma^2$ of the underlying distribution is finite. I would guess that you are using a parameterization of the Pareto for which $\sigma^2 = \infty$, in which case the CLT will not hold. If you tweak your scale parameter so that $\sigma^2 < \infty$ you should see the result you want.

  • $\begingroup$ Thank you for the quick response! I am going to try your proposal and let you know. :) $\endgroup$
    – yaskovdev
    Jun 17 '18 at 17:17
  • $\begingroup$ I have updated the code in the question, now it is exactly one I am running. If the parameter b is 3 (one I'm using), then the mean and the variance are 1.5 and 0.75, respectively. So that's unlikely the reason. By the way, if I set b to 1, then the mean and the variance are both infinity, and then, indeed, the histogram looks completely different from the expected "normal" shape. :) $\endgroup$
    – yaskovdev
    Jun 18 '18 at 9:27
  • 1
    $\begingroup$ You may even try upping the same size to see it more clearly. The speed of convergence of the CLT depends on how non-normal the underlying distribution is, and the Pareto with its heavy tail may take more time than better-behaved distributions. $\endgroup$
    – dsaxton
    Jun 18 '18 at 13:20
  • 1
    $\begingroup$ Try looking at the means of 1000 bunches of 1000 values... $\endgroup$
    – Spacedman
    Jun 18 '18 at 14:02
  • $\begingroup$ I've increased n (even to 5000), and, as a result, I have finally got the expected graphs: github.com/yaskovdev/data-science-sandbox/blob/master/…. Pareto distribution indeed requires much bigger sample size to start looking as the normal distribution. Thanks to everyone for the help! $\endgroup$
    – yaskovdev
    Jun 19 '18 at 4:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.