Auto-Encoder customized layer training

Question

My question is related with model-weights optimization during back propagation. In this image I'm trying to represent an auto-encoder having 7 layers where 4th one is center layer.

If my understanding is correct, during back propagation outcome of jth+1 layer, will help in optimizing the weights of jth layer. If it is correct, I want to restrict the weights training of Encoder layer and want to perform my customized training.

Customized training should be like this:

1. Till layer 4 normal back propagation.
2. Layer 3 weights will get trained(optimized) using the output of 4th layer as mentioned in PINK arrow.
3. Layer 2 weights will get trained with the outcome of layer 5 as mentioned in YELLOW color arrow.
4. Layer 1 weights will get trained with the outcome of layer 6 as mentioned in BLUE color.

I want to know how I can implement such customized trainable architecture?I'm familiar with Tensorflow, but I'm not able to figure out any solution and kind of stuck.

NOTE: Bear with my drawing.

Answer 1

You have to create multiple training sessions, in which you selectively freeze the layers you don't want to train. Each Sequential() model can be seen as a list of layer objects. Each of these layers have the argument trainable = True/False. You Freeze the layers you don't want to be trained classically, and proceed to

Then, using Keras Models, you connect layers that are not adjacent as you depicted in the picture. Once all layers have been trained, you transfer the weights into a news Sequential() shell to compose a final model.

It's going to be time consuming, but it's doable.

Answer 2

The definition of your customized training is too vague to be able to implement it: what is the "outcome" of a layer? How do you "train a layer with the outcome of another layer"?

Also, note that, while it is possible to freeze layers selectively during training (i.e. not applying any update during the optimization step), the gradients will always propagate backward through layers that are connected.

Nevertheless, there is a viable approach that may be somewhat similar to the intuition behind your proposal: residual connections.

As shown in the figure above (taken from the original paper), residual connections are simply connections from early layers to further points in the network by simply adding them. They allow gradients to be more easily propagated.

You could create your autoencoder so that the result of each decoder layer gets added the result of the encoder layer of matching size.