Can TF turn a given graph into a recursive one?

Question

I have a complex graph that takes an input $x_t$, it does a bunch of convolutions, tensor shape manipulations etc... until it produces $y_t$. I want now to try a different approach, where I feed $y_t$ at some point into the graph (I concatenate it to some features after a convolution), while the net is processing $x_{t+1}$.

So the net should do the following ($y_0$ is a fixed initial input):

$(x_0,y_0)\rightarrow y_1$

$(x_1,y_1)\rightarrow y_2$

etc...

I have been looking at the standard RNN implementations in TensorFlow (GRU, LSTM, etc), but I don't quite understand how I can do this, because it seems that LSTM and the other RNN units have a fixed graph (like, a fully connected layer) and I cannot use the one that I need instead.

Do I need to subclass tf.nn.rnn_cell.RNNCell? That seems problematic too because it seems to admit only 2D tensors as input.

EDIT: I just discovered this awesome article and I'm going to document myself about tf.scan.

Answer 1

I finally managed to find a solution by using tf.scan (see this question on stackexchange)

The trick is to create a model object that calls tf.scan in one of its methods using a step function that operates on the elements of a list of inputs.

The function tf.scan takes care of feeding the output of the step function back into itself for the next iteration, and it is quite flexible so you might be able to do what you need without the need to implement a full RNN from scratch (say, with tf.nn.raw_rnn).

Below, the minimal working example that I had posted over there:

import tensorflow as tf
import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2' # to avoid TF suggesting SSE4.2, AVX etc...

class Model():
    def __init__(self):
        self._inputs = tf.placeholder(shape=[None], dtype=tf.float32)
        self._predictions = self._compute_predictions()

    def _step(self, old_state, new_input):
        # ---- In here I will write a much more complex graph ----
        return old_state + new_input

    def _compute_predictions(self):
        return tf.scan(self._step, self._inputs, initializer = tf.Variable(0.0))

    @property
    def predictions(self):
        return self._predictions

    @property
    def inputs(self):
        return self._inputs

def test(sess, model):
    sess.run(tf.global_variables_initializer())
    print(sess.run(model.predictions, {model.inputs: [1.0, 2.0, 3.0, 4.0]}))

test(tf.Session(), Model())