1
$\begingroup$

I have many data in the form of one dimensional histogram, to give an example consider the data at http://pastebin.com/embed_js.php?i=1mNRuEHZ

histogram test

I expect that these data are obtained from a pdf composed of a sum of Gaussian distributions but with their actual number being unknown.

I want then to fit these data with a model given by a sum of Gaussians

$$ f_N(x) = \sum_{k=1}^N c_k \exp \left[ - \frac{(x-a_k)^2}{2 b_k^2} \right] $$

where $a_k$, $b_k$, $c_k$ and $N$ are in principle parameters to be fitted.

The problem can be viewed as a problem of model selection between different $f_N$ for all the possible values of $N$.

My idea is then to make an usual fit by least squares at fixed $N$ and then compare the results for different $N$ via some statistical measure of the quality of the fit, as e.g. Akaike information criterion (AIC). Is this an acceptable test?

For the data of the previous example (that I generated as a test from 3 gaussians) I will obtain

 N        AIC
 -------------
 1      +568.1
 2      +557.4
 3      -446.6
 4      -443.5
 5      -442.7

As you see, the AIC decreases quickly till $N=3$, where it starts to increase again very slowly (due to overfitting). So it is clear that $N=3$ is the best choice here.

I mention that in some cases I have more data histograms extracted from the same pdf, so I could do some cross validation. Although this always a good test, the availability of my data changes from sample to sample, so I'd prefer to have a criterion for the single histogram (if the quality of the test is not too much different).

Improve this question
$\endgroup$
10
  • 1
    $\begingroup$ Perhaps you could explain a bit more what your data looks like? How many dimensions, etc? Do you have (x,y) data and you are trying to fit a curve through the points, or is it a density you are trying to match? $\endgroup$
    – Spacedman
    Commented Sep 7, 2015 at 16:29
  • 2
    $\begingroup$ You don't have histograms nor do you have random samples from a mixture of normal distributions. (If so, you certainly wouldn't be getting negative numbers for the counts or for a probably density.) It appears that you have samples from a fixed set of x's where the expected values for the vertical axis are from a linear combination of curves with a Gaussian shape and you've added random noise from a separate normal distribution. I don't mean to be so negative and I'll be more constructive in my next comment. $\endgroup$
    – JimB
    Commented Sep 11, 2015 at 1:29
  • 1
    $\begingroup$ But N=3 does have the smallest AIC value. -446.6 < -443.5 < -442.7. I also wonder if you've restricted the values of c to be non-negative. $\endgroup$
    – JimB
    Commented Sep 11, 2015 at 5:22
  • 1
    $\begingroup$ You may have a look at a similar question I answered at Mathematica Stack Exchange: bit.ly/1XW2Iob. $\endgroup$
    – Romke Bontekoe
    Commented Sep 12, 2015 at 7:54
  • 1
    $\begingroup$ Just a small caution: when fitting with linear combinations of gaussian curves, the number of peaks can be less than the number of gaussian curves. There are a variety of automated methods to find peaks (although none are perfect in part because a peak depends on the scale that you define what a peak is). You might google for "bump hunting" and "peaks" in this forum and the related Cross Validation and Mathematica forums. An example with Mathematica is at mathematica.stackexchange.com/questions/23828/…. $\endgroup$
    – JimB
    Commented Sep 13, 2015 at 18:09

1 Answer 1

Reset to default
1
$\begingroup$

You're almost done with your results. Try to substitute the AIM for the Likelihood of the sum of Gaussian with your data, and as soon as your Likelihood value stabilizes and doesn't increase with the number of Gaussians you've found the right value for your N.

You can even try to look for the EM algorithm for the estimation of the Mixture of gaussians that fit your data.

The value of L = f(N) increases as a log function and the so called "knee" is where your optimal value of N lies on!

Improve this answer
$\endgroup$
2
  • $\begingroup$ thanks. Also, as noted by @Jim_Baldwin, indeed the AIC is indeed minimum at N=3, so it already gives a good criterion to fix its value. $\endgroup$
    – psmith
    Commented Sep 11, 2015 at 16:37
  • 1
    $\begingroup$ That's right! It's basically the same approach! Good work @psmith! $\endgroup$
    – Ricardo Domingos Ferreira
    Commented Sep 12, 2015 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.