2
$\begingroup$

After studying some academic papers about neural network recognition, plus trying some of them out myself, I do understand how you can train a network to give you a certain output on a defined set of output neurons depending on the input.

However I struggle to understand, how I can dynamically add output neurons. I've seen neural network applications, which where recognizing objects from images, however they set of classes to be recognized could always be increased. From my understanding this should not be possible, as the size of output neurons is fixed.

Lets assume I want to build a application in which user can make a photograph, upload it to the database and use if subsequent for image recognition. I see only two possible solutions:

Retraining

Retrain the whole neural network every time some adds a new image. However training is very expensive and this would require a lot of resources so users can add all their objects.

"Dead" Outputs

as suggested in https://datascience.stackexchange.com/a/9551/18744

We could add a lot of neurons to the output-layer which are always trained to 0, unless we need more outputs. Once a new object appears, we start training the new output. However I believe the performance of such procedure would suffer in recognition terms, as we leave important training feature such as randomization out.

$\endgroup$

1 Answer 1

1
$\begingroup$

Modifying the output layer of neural networks is part of transfer learning.

Adding new classes/output nodes always requires additional training.

One common option is just to retrain the last layer, aka the softmax layer. The last layer is the mapping from feature extractors to category membership.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.