0
$\begingroup$

The AIC formula is :

$AIC = 2k + n Log(RSS/n)$

So if RSS is equal to 0, it is undefined. How do I deal with this? What value should it take?

$\endgroup$
1
  • $\begingroup$ So your model has a perfect fit? $\endgroup$
    – Peter
    Oct 19 '20 at 10:49
1
$\begingroup$

RSS = 0 implies that the model is a perfect fit since there was no residual. The limit of the log of 0 is $-\infty$, and since lower AICs are better, and this model is perfect,it makes perfect sense that the AIC should be a negative number such that no number can be lower.

$\endgroup$
7
  • $\begingroup$ Right, that makes sense, but the problem is that I am evaluating the AIC fo each fit and then I want to take the mean of these values (I cannot compute the AIC of the model with all the fits together because I have a different number of datapoints for each). So how should I handle this? $\endgroup$
    – user606273
    Oct 19 '20 at 9:53
  • $\begingroup$ I would recommend opening a new question regarding taking the mean of an AIC across many fits, and how to deal with the case that a given fit is perfect. It's not something that I've heard of before, so I think you would have to provide lots of details and particularly why you're doing it. $\endgroup$ Oct 19 '20 at 9:56
  • $\begingroup$ Maybe my approach of taking the mean is wrong. Do you know how could I compute the AIC if for every fit I have a different number of data points? $\endgroup$
    – user606273
    Oct 19 '20 at 10:01
  • $\begingroup$ As far as I know, you can't compare the AIC between models on different datasets, that's where my understanding of what you're trying to achieve falls apart. If you have many models (or fits), there's nothing to stop you from calculating the AIC of each one. $\endgroup$ Oct 19 '20 at 10:31
  • $\begingroup$ Agreed, on different datasets you cannot (well, you actually can but it takes som transformations) $\endgroup$
    – CutePoison
    Oct 19 '20 at 10:42
0
$\begingroup$

As a follow-up to @Cameron Chandler; if your RSS=0 it should trigger some alerts since it is 100% overfitting, thus evaluating that model does not make any sense as such. But yes, take the limit of $\log(x), x\rightarrow 0^+$ and notice it goes towards $-\infty$ thus you cannot find a more perfekt model - if you only look at AIC, but you can (almost) not find a worse model if you look at the application. If you evaluate multiple fits I would simply just ignore those "perfect fits"

$\endgroup$
9
  • $\begingroup$ As I mentioned in the comment above, I am evaluating the AIC for each fit and then I want to take the mean of these values, so if I set it to minus infinity, I will have problems with the mean whenever I have one perfect fit in my dataset $\endgroup$
    – user606273
    Oct 19 '20 at 9:56
  • $\begingroup$ I have edited the answer accordingly $\endgroup$
    – CutePoison
    Oct 19 '20 at 10:02
  • $\begingroup$ The thing is it will be a perfect fit for one of the models but not for the rest, so if I just ignore it, I would be loosing information. Maybe I should just set it to a big negative number? $\endgroup$
    – user606273
    Oct 19 '20 at 10:54
  • $\begingroup$ How would you lose information? I would treat it as "missing" since the score provides no information what so ever (other than you are overfitting) $\endgroup$
    – CutePoison
    Oct 19 '20 at 10:58
  • $\begingroup$ Ah I see your point, but I am not trying to evaluate overfitting at the moment since I am fitting the parameters for each tumor. But maybe it is the best solution, thanks. $\endgroup$
    – user606273
    Oct 19 '20 at 11:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.