1
$\begingroup$

I have many data in the form of one dimensional histogram, to give an example consider the data at http://pastebin.com/embed_js.php?i=1mNRuEHZ

histogram test

I expect that these data are obtained from a pdf composed of a sum of Gaussian distributions but with their actual number being unknown.

I want then to fit these data with a model given by a sum of Gaussians

$$ f_N(x) = \sum_{k=1}^N c_k \exp \left[ - \frac{(x-a_k)^2}{2 b_k^2} \right] $$

where $a_k$, $b_k$, $c_k$ and $N$ are in principle parameters to be fitted.

The problem can be viewed as a problem of model selection between different $f_N$ for all the possible values of $N$.

My idea is then to make an usual fit by least squares at fixed $N$ and then compare the results for different $N$ via some statistical measure of the quality of the fit, as e.g. Akaike information criterion (AIC). Is this an acceptable test?

For the data of the previous example (that I generated as a test from 3 gaussians) I will obtain

 N        AIC
 -------------
 1      +568.1
 2      +557.4
 3      -446.6
 4      -443.5
 5      -442.7

As you see, the AIC decreases quickly till $N=3$, where it starts to increase again very slowly (due to overfitting). So it is clear that $N=3$ is the best choice here.

I mention that in some cases I have more data histograms extracted from the same pdf, so I could do some cross validation. Although this always a good test, the availability of my data changes from sample to sample, so I'd prefer to have a criterion for the single histogram (if the quality of the test is not too much different).

$\endgroup$
  • 1
    $\begingroup$ Perhaps you could explain a bit more what your data looks like? How many dimensions, etc? Do you have (x,y) data and you are trying to fit a curve through the points, or is it a density you are trying to match? $\endgroup$ – Spacedman Sep 7 '15 at 16:29
  • 2
    $\begingroup$ You don't have histograms nor do you have random samples from a mixture of normal distributions. (If so, you certainly wouldn't be getting negative numbers for the counts or for a probably density.) It appears that you have samples from a fixed set of x's where the expected values for the vertical axis are from a linear combination of curves with a Gaussian shape and you've added random noise from a separate normal distribution. I don't mean to be so negative and I'll be more constructive in my next comment. $\endgroup$ – JimB Sep 11 '15 at 1:29
  • 1
    $\begingroup$ But N=3 does have the smallest AIC value. -446.6 < -443.5 < -442.7. I also wonder if you've restricted the values of c to be non-negative. $\endgroup$ – JimB Sep 11 '15 at 5:22
  • 1
    $\begingroup$ You may have a look at a similar question I answered at Mathematica Stack Exchange: bit.ly/1XW2Iob. $\endgroup$ – Romke Bontekoe Sep 12 '15 at 7:54
  • 1
    $\begingroup$ Just a small caution: when fitting with linear combinations of gaussian curves, the number of peaks can be less than the number of gaussian curves. There are a variety of automated methods to find peaks (although none are perfect in part because a peak depends on the scale that you define what a peak is). You might google for "bump hunting" and "peaks" in this forum and the related Cross Validation and Mathematica forums. An example with Mathematica is at mathematica.stackexchange.com/questions/23828/…. $\endgroup$ – JimB Sep 13 '15 at 18:09
1
$\begingroup$

You're almost done with your results. Try to substitute the AIM for the Likelihood of the sum of Gaussian with your data, and as soon as your Likelihood value stabilizes and doesn't increase with the number of Gaussians you've found the right value for your N.

You can even try to look for the EM algorithm for the estimation of the Mixture of gaussians that fit your data.

The value of L = f(N) increases as a log function and the so called "knee" is where your optimal value of N lies on!

$\endgroup$
  • $\begingroup$ thanks. Also, as noted by @Jim_Baldwin, indeed the AIC is indeed minimum at N=3, so it already gives a good criterion to fix its value. $\endgroup$ – psmith Sep 11 '15 at 16:37
  • 1
    $\begingroup$ That's right! It's basically the same approach! Good work @psmith! $\endgroup$ – Ricardo Domingos Ferreira Sep 12 '15 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.