Timeline for Does keras categorical_cross_entropy loss take incorrect classification into account
Current License: CC BY-SA 3.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 16, 2020 at 11:08 | history | edited | CommunityBot |
Commonmark migration
|
|
| Dec 23, 2017 at 19:09 | comment | added | Green Falcon | @NeilSlater sure :) | |
| Dec 23, 2017 at 14:46 | comment | added | Neil Slater | @Media: I'm not clear about your example. Maybe ask a question about it showing the full vector i.e. the whole label, and the whole estimate where you think there is a problem. | |
| Dec 23, 2017 at 14:04 | comment | added | Green Falcon | @NeilSlater actually I was thinking about something. In the case that I mentioned, the error is zero which is the least error value that exist although there is 100% error in the typical mentioned situation, back propagation is leading us where? zero error rate isn't optimal? If gradient descent tries to move us to better place, Is there other places better than zero error? because this cost function can not be negative. | |
| Dec 23, 2017 at 9:19 | comment | added | Neil Slater | @Media: Yes it will get updated, because the softmax transform links all the neuron values together, so even though the loss is effectively only calculated on one neuron's output after the softmax is applied, you will still calculate a non-zero gradient value for all the neurons at the pre-softmax linear (logit) stage using back propagation. | |
| Dec 23, 2017 at 8:00 | comment | added | Green Falcon | based on your answer here and as I know, the second formula should be used for binary tasks to avoid the problem I mentioned. | |
| Dec 23, 2017 at 7:47 | comment | added | Green Falcon | @NeilSlater actually I always had this problem but did not know how to ask. Suppose that you have just one sample and the label for that should be zero but the output of softmax is non-zero. Does it ever get updated? the lost is always zero. | |
| Dec 23, 2017 at 6:25 | comment | added | Jason Davis | Thanks, I realize the missing link in my logic was that I was using the softmax activation, where the sum of the probabilities adds up to 1, and so each wrong classification takes away from the "strength" of the correct classification. | |
| Dec 23, 2017 at 6:01 | vote | accept | Jason Davis | ||
| Dec 22, 2017 at 9:37 | history | edited | Neil Slater | CC BY-SA 3.0 |
deleted 8 characters in body
|
| Dec 22, 2017 at 9:30 | history | edited | Neil Slater | CC BY-SA 3.0 |
added 699 characters in body
|
| Dec 22, 2017 at 9:13 | history | answered | Neil Slater | CC BY-SA 3.0 |