Can neural networks be adapted without recreating them completely?

Question

If I have, for example, a classification network which can tell if there is a dog or a cat in a picture, is it possible to adapt the network so it can also learn to detect a mouse? Without making a new one from scratch. In this case it doesn't make sense, but I'm wondering if for example Netflix has to retrain its network completely with every new show they add. And if so, when do they do that? The first few days may be the most crucial ones, but also the ones with the least data to train a network from.

The actual problem I do have is that I'm trying to train a network to predict the location of public transport vehicles. That by itself is hard enough, but what do I do if there is a new transport line, which I don't have an input neuron for. - Here I would need to add an input neuron.

Or another idea I'm thinking about is a neural network which can help me order my documents by detecting the company and date. I would like to classify an unknown document layout, a few times, so the neural network is able to classify it by itself, but without losing everything it learned so far. - Here I would need to add an output neuron.

It seems like there is a way to do this, since there are some machine learning algorithms out there which seem to pull off stuff like that. Or do they somehow work around this? If so, how?

Answer 1

The answer to your question is "Transfer Learning".

Since the datasets "cat and dog" and "mouse" are quite similar as both are images. In DeepNet for recognising "cats and dogs", any deep learning network in its early layers learn to identify low-level features like edges, etc. It learns high-level feature like eyes, ears, etc in few further layers and in very later layers of DeepNet, it starts to recognise the intended object. In DeepNet for recognising "dogs", similar pattern will follow. On comparing these two, one may find that first few layers of DeepNet in both cases produce similar low-level features like edges, etc.

Hence, first few layers of the DeepNet learned from "cats" can be used as base layers for the DeepNet for "mouse" detection. This technique is called transfer learning.

To apply transfer learning, dataset on which DeepNet has been trained on and dataset on which this technique can be applied must be similar.

This transfer learning video by Andrew Ng would also be helpful in understanding the concept

Answer 2

When I comes to adding a new output, you need only to retrain the last layer, it's called transfer learning. And pretty good example of that is retraining the last layer of one of the inception modules of google (Originally trained on the ImageNet dataset) to classify flowers. You can find the example on the website of Tensorflow.

When it comes to adding and input neuron, i never encountered anything of the sort to be honest.