I have a partial neural network with several layers of various types, with weights $\theta$; let's call it $F_{\theta}$; and the dimensions of input and output arrays are the same, implemented in Keras with Tensorflow backend. Within this framework, how do train a model of the form $y = F_{\theta}(F_{\theta}(x))$, or more generally some $n$-th iteration of $F_{\theta}$? In other words, I reuse an entire subset of the neural network, with all the same weights, so the full model is deeper but does not have more parameters than the shallower model.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
When you build your Keras model using the functional interface, you can also build additional models on any subset of the paths through the network by reusing the intermediary functions. Then you can train on just portions of the network (given that you have targets for the outputs). I haven't tried to train on sub-networks of a network, but I do use these intermediary models to propagate activations between internal layers. The Python package conx that is built on top of Keras will build these intermediary models for you, by the way. For more, see: http://conx.readthedocs.io/en/latest/Getting%20Started%20with%20conx.html?highlight=propagate_from#Propagation-functions
