I am new to Data Science. Recently I was studying a course about statistics. One of the tasks there was to check the central limit theorem in practice.

The idea was quite simple: take a continuous random variable; generate, say, 1000 samples from it, each of size n; draw a histogram of the samples. Then find the parameters of the normal distribution using the central limit theorem and draw the PDF of the distribution. As a result, the histogram and the PDF should be, roughly speaking, "similar" (and become more "similar" as n grows).

I chose Pareto distribution and, with this Python code,

import numpy as np
import matplotlib.pyplot as plt
import scipy.stats as sts
from math import sqrt

n = 10

b = 3.0
random_value = sts.pareto(b)
mean = random_value.mean()
variance = random_value.var()

plt.hist(samples(random_value, n, 1000), bins=20, normed=True)

x = np.linspace(0, 6, 100)
pdf = sts.norm(mean, sqrt(variance / n)).pdf(x)
plt.plot(x, pdf, color='r', label='theoretical PDF')

where the samples function was like this

def samples(random_value, sample_size, number_of_samples):
    result = np.asarray([])
    for i in range(number_of_samples):
        result = np.append(result, np.mean(random_value.rvs(sample_size)))
    return result

I got the next histogram and PDF as a result Pareto distribution

The result looked suspicious to me: there was no expected similarity. I checked the same code with another continuous distribution (with a uniform distribution), and the graphs looked much more similar.

Is it a feature of the Pareto distribution which makes it a bad choice to demonstrate how the central limit theorem works? Or something is wrong with the code itself (e.g., the parameters of the normal distribution are calculated in a wrong way)? Thank you in advance.

  • 3
    $\begingroup$ I'm voting to close this question as off-topic because there's better statistical expertise at crossvalidated: stats.stackexchange.com $\endgroup$
    – Spacedman
    Jun 17 '18 at 22:02
  • 1
    $\begingroup$ where does random_value.rvs come from? Please try and make a reproducible example. $\endgroup$
    – Spacedman
    Jun 17 '18 at 22:10
  • $\begingroup$ @Spacedman, thank you for the hint. I have updated the code, now it is exactly the same I am executing. $\endgroup$
    – yaskovdev
    Jun 18 '18 at 9:34

One key assumption of the central limit theorem is that the variance $\sigma^2$ of the underlying distribution is finite. I would guess that you are using a parameterization of the Pareto for which $\sigma^2 = \infty$, in which case the CLT will not hold. If you tweak your scale parameter so that $\sigma^2 < \infty$ you should see the result you want.

  • $\begingroup$ Thank you for the quick response! I am going to try your proposal and let you know. :) $\endgroup$
    – yaskovdev
    Jun 17 '18 at 17:17
  • $\begingroup$ I have updated the code in the question, now it is exactly one I am running. If the parameter b is 3 (one I'm using), then the mean and the variance are 1.5 and 0.75, respectively. So that's unlikely the reason. By the way, if I set b to 1, then the mean and the variance are both infinity, and then, indeed, the histogram looks completely different from the expected "normal" shape. :) $\endgroup$
    – yaskovdev
    Jun 18 '18 at 9:27
  • 1
    $\begingroup$ You may even try upping the same size to see it more clearly. The speed of convergence of the CLT depends on how non-normal the underlying distribution is, and the Pareto with its heavy tail may take more time than better-behaved distributions. $\endgroup$
    – dsaxton
    Jun 18 '18 at 13:20
  • 1
    $\begingroup$ Try looking at the means of 1000 bunches of 1000 values... $\endgroup$
    – Spacedman
    Jun 18 '18 at 14:02
  • $\begingroup$ I've increased n (even to 5000), and, as a result, I have finally got the expected graphs: github.com/yaskovdev/data-science-sandbox/blob/master/…. Pareto distribution indeed requires much bigger sample size to start looking as the normal distribution. Thanks to everyone for the help! $\endgroup$
    – yaskovdev
    Jun 19 '18 at 4:25

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .