# Fittting histograms with combination of an unknown number of Gaussians

I have many data in the form of one dimensional histogram, to give an example consider the data at http://pastebin.com/embed_js.php?i=1mNRuEHZ

I expect that these data are obtained from a pdf composed of a sum of Gaussian distributions but with their actual number being unknown.

I want then to fit these data with a model given by a sum of Gaussians

$$f_N(x) = \sum_{k=1}^N c_k \exp \left[ - \frac{(x-a_k)^2}{2 b_k^2} \right]$$

where $a_k$, $b_k$, $c_k$ and $N$ are in principle parameters to be fitted.

The problem can be viewed as a problem of model selection between different $f_N$ for all the possible values of $N$.

My idea is then to make an usual fit by least squares at fixed $N$ and then compare the results for different $N$ via some statistical measure of the quality of the fit, as e.g. Akaike information criterion (AIC). Is this an acceptable test?

For the data of the previous example (that I generated as a test from 3 gaussians) I will obtain

 N        AIC
-------------
1      +568.1
2      +557.4
3      -446.6
4      -443.5
5      -442.7


As you see, the AIC decreases quickly till $N=3$, where it starts to increase again very slowly (due to overfitting). So it is clear that $N=3$ is the best choice here.

I mention that in some cases I have more data histograms extracted from the same pdf, so I could do some cross validation. Although this always a good test, the availability of my data changes from sample to sample, so I'd prefer to have a criterion for the single histogram (if the quality of the test is not too much different).

• Perhaps you could explain a bit more what your data looks like? How many dimensions, etc? Do you have (x,y) data and you are trying to fit a curve through the points, or is it a density you are trying to match? Sep 7 '15 at 16:29
• You don't have histograms nor do you have random samples from a mixture of normal distributions. (If so, you certainly wouldn't be getting negative numbers for the counts or for a probably density.) It appears that you have samples from a fixed set of x's where the expected values for the vertical axis are from a linear combination of curves with a Gaussian shape and you've added random noise from a separate normal distribution. I don't mean to be so negative and I'll be more constructive in my next comment.
– JimB
Sep 11 '15 at 1:29
• But N=3 does have the smallest AIC value. -446.6 < -443.5 < -442.7. I also wonder if you've restricted the values of c to be non-negative.
– JimB
Sep 11 '15 at 5:22
• You may have a look at a similar question I answered at Mathematica Stack Exchange: bit.ly/1XW2Iob. Sep 12 '15 at 7:54
• Just a small caution: when fitting with linear combinations of gaussian curves, the number of peaks can be less than the number of gaussian curves. There are a variety of automated methods to find peaks (although none are perfect in part because a peak depends on the scale that you define what a peak is). You might google for "bump hunting" and "peaks" in this forum and the related Cross Validation and Mathematica forums. An example with Mathematica is at mathematica.stackexchange.com/questions/23828/….
– JimB
Sep 13 '15 at 18:09