After studying some academic papers about neural network recognition, plus trying some of them out myself, I do understand how you can train a network to give you a certain output on a defined set of output neurons depending on the input.
However I struggle to understand, how I can dynamically add output neurons. I've seen neural network applications, which where recognizing objects from images, however they set of classes to be recognized could always be increased. From my understanding this should not be possible, as the size of output neurons is fixed.
Lets assume I want to build a application in which user can make a photograph, upload it to the database and use if subsequent for image recognition. I see only two possible solutions:
Retrain the whole neural network every time some adds a new image. However training is very expensive and this would require a lot of resources so users can add all their objects.
as suggested in https://datascience.stackexchange.com/a/9551/18744
We could add a lot of neurons to the output-layer which are always trained to 0, unless we need more outputs. Once a new object appears, we start training the new output. However I believe the performance of such procedure would suffer in recognition terms, as we leave important training feature such as randomization out.