Prediction on timeseries data using tensorflow

Question

I have an input and output of below format:

(X) = [[ 0  1  2]
       [ 1  2  3]] 

  y   = [ 3  4 ]

It's timeseries data. The task is to predict the next number. Basically, the input was crafted by the below snippet:

 def split_sequence(arr,timesteps):
     arr_len = len(arr)
     X,y = [],[]
     for i in range(arr_len):
       end_idx = i + timesteps
       if end_idx > arr_len-1:
          break
       input_component,output_component =  arr[i:end_idx],arr[end_idx]
       X.append(input_component)
       y.append(output_component)

     return np.array(X), np.array(y)

Now, I would like to train the model on the input and predict the next number. For instance, x = [81,82,83] and the predicted output would be y = 84. I learned how to do this in keras. However, I would like to try to do this in tensorflow as well.

Below is the code in tensorflow:

 # Data generator
 def generate_batch(X,y,batch_size):
    m = X.shape[0]
    indexes = range(m)
    n_batches = m // batch_size
    for batch_index in np.array_split(indexes,n_batches):
       yield X[batch_index],y[batch_index] 

 # parameters
 n_inputs = 3
 n_epochs = 1000
 batch_size = 40
 learning_rate = 0.01
 n_steps = 3

 # generate the input and output using split_sequence method
 input, output = split_sequence(range(1000),n_steps)


 # Define the input variables
 X = tf.placeholder(tf.int32,shape=(None,n_inputs),name='X')
 y = tf.placeholder(tf.float32,shape=(None),name='y')
 theta = tf.Variable(tf.random_uniform([n_steps,1],-1.0,1.0),name='theta')

 # predictions and error 
 y_predictions = tf.matmul(X,theta,name='predictions')
 error = y_predictions - y
 mse = tf.reduce_mean(tf.square(error),name='mse')



 # train the model          
 optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
 training_op = optimizer.minimize(mse)

 init = tf.global_variables_initializer()

 with tf.Session() as session:      
   session.run(init)
   for epoch in range(n_epochs):
      for X_batch,y_batch in generate_batch(input,output,batch_size):
         if epoch % 10 == 0:
             print('epoch',epoch,'MSE=',mse.eval())
         session.run(training_op,feed_dict={X:X_batch,y:y_batch})

To be honest, I am completely stuck with the below error:

You must feed a value for placeholder tensor 'X' with dtype float and shape [?,3].

My input is an integer, so that was the reason behind defining:

 X = tf.placeholder(tf.int32,shape=(None,n_inputs),name='X')

Can someone help me fix this? Also, if I wanted to add bias variable will I be able to achieve for the above input?

Answer 1

The error is caused by this line:

print('epoch',epoch,'MSE=',mse.eval())

This happens because the tensor mse also depends on the placeholders X and y. One way to fix this would be to change the training loop to be:

for X_batch,y_batch in generate_batch(input,output,batch_size):
     mse_val, _ = session.run([mse, training_op],feed_dict={X:X_batch,y:y_batch})
     if epoch % 10 == 0:
         print('epoch',epoch,'MSE=',mse_val)

Also you will need to switch X back to tf.float32 since tf.matmul is not compatible with int and float. The data will automatically be casted once you feed it in.

To add a bias variable, you can do it similarly to how you define theta.

b = tf.Variable(0.0, dtype=tf.float32, name='b')
...
y_predictions += b