1
$\begingroup$

I am working on ANN. I have 2497 training examples and each of them is a vector of 128. So the input size is 128. Number of neurons in hidden layer is 64 and number of output neurons is 6 (since classes are six). My Target vector looks something like this : [0 1 0 0 0 0]. This means that the example belongs to class 2. I have used sigmoid as an activation at all layers and sum of squared error is loss. SSE is computed over one epoch. Total epochs are 10k. my loss starts from around 700 and reduces to 450. Should I say that loss is 18% per example since 450 is the loss for all the 2497 examples. How do I interpret this. Is my model good enough? I know that I should test it on unseen data to be sure of its accuracy but still does this tell anything about the performance at all or not.

ps: I am implementing it in C

$\endgroup$
  • $\begingroup$ In the last layer, the one with many neurons as classes, you have to use softmax activation $\endgroup$ – Francesco Pegoraro Oct 3 '18 at 13:00
  • $\begingroup$ How exactly are you computing SSE in a classification task? Can you post the formula? $\endgroup$ – shadowtalker Oct 3 '18 at 16:19
0
$\begingroup$

SSE in classification is proportional to the Brier score:

$$ SSE = \sum_n \sum_k (\hat y_{nk} - y_{nk})^2 \\ Brier = \frac{SSE}{N} $$

For observations indexed by $n$ and classes indexed by $k$.

In your case, you obtained a Brier score of 0.18.

The Brier score can be decomposed into

$$ Brier = Reliability - Resolution + Uncertainty $$

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.