I am new to Tensorflow. I have trained Tensorflow model, but I need to take model predictions and add them to my original test set as a column. How can I do that?

def model(self, layers_dims, X_train, Y_train, X_test, Y_test, learning_rate=0.00001,
          num_epochs=1000, print_cost=True):
    Implements a three-layer tensorflow neural network: LINEAR->RELU->LINEAR->RELU->LINEAR->SOFTMAX.

    X_train -- training set, of shape (input size = 12288, number of training examples = 1080)
    Y_train -- test set, of shape (output size = 6, number of training examples = 1080)
    X_test -- training set, of shape (input size = 12288, number of training examples = 120)
    Y_test -- test set, of shape (output size = 6, number of test examples = 120)
    learning_rate -- learning rate of the optimization
    num_epochs -- number of epochs of the optimization loop
    minibatch_size -- size of a minibatch
    print_cost -- True to print the cost every 100 epochs

    parameters -- parameters learnt by the model. They can then be used to predict.

    ops.reset_default_graph()  # to be able to rerun the model without overwriting tf variables
    tf.set_random_seed(1)  # to keep consistent results
    seed = 3  # to keep consistent results
    (n_x, m) = X_train.shape  # (n_x: input size, m : number of examples in the train set)

    n_y = Y_train.shape[0]  # n_y : output size
    #print('Ytrain shape', Y_train.shape)
    costs = []  # To keep track of the cost

    # Create Placeholders of shape (n_x, n_y)
    X, Y = self.create_placeholders(n_x, n_y)

    # Initialize parameters
    parameters = NN_predict_trading_decisions.initialize_parameters(layers_dims)

    # Forward propagation: Build the forward propagation in the tensorflow graph
    ZL = NN_predict_trading_decisions.forward_propagation(X, parameters)

    # Cost function: Add cost function to tensorflow graph
    cost = NN_predict_trading_decisions.compute_cost(ZL, Y)

    # Backpropagation: Define the tensorflow optimizer. Use an AdamOptimizer.
    #optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
    optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)

    # Initialize all the variables
    init = tf.global_variables_initializer()

    # Start the session to compute the tensorflow graph
    with tf.Session() as sess:

        # Run the initialization

        # Do the training loop
        for epoch in range(num_epochs):

            epoch_cost = 0.  # Defines a cost related to an epoch
            current_cost = sess.run([optimizer, cost], feed_dict= {X: X_train, Y: Y_train})
            epoch_cost +=  current_cost[1]

            # Print the cost every epoch
            if print_cost == True and epoch % 100 == 0:
                print("Cost after epoch %i: %f" % (epoch, epoch_cost))
            if print_cost == True and epoch % 5 == 0:

        # plot the cost
        plt.xlabel('iterations (per tens)')
        plt.title("Learning rate =" + str(learning_rate))

        # lets save the parameters in a variable
        parameters = sess.run(parameters)
        print("Parameters have been trained!")

        # Calculate the correct predictions
        correct_prediction = tf.equal(tf.argmax(ZL), tf.argmax(Y))

        # Calculate accuracy on the test set
        accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

        print("Train Accuracy:", accuracy.eval({X: X_train, Y: Y_train}))
        print("Test Accuracy:", accuracy.eval({X: X_test, Y: Y_test}))

        return parameters
  • $\begingroup$ Save output in a list or numpy array. $\endgroup$
    – Ankit Seth
    Sep 5 '18 at 10:29
  • $\begingroup$ How can I do that? Sorry, I am very new) $\endgroup$
    – Myron
    Sep 5 '18 at 10:34
  • $\begingroup$ Can you post some part of your code where you are getting predictions after training? $\endgroup$
    – Ankit Seth
    Sep 5 '18 at 10:38
  • $\begingroup$ So, I did a little research and found this post, it would be useful for you. stackoverflow.com/questions/40002084/… $\endgroup$
    – Ankit Seth
    Sep 5 '18 at 14:35
  • $\begingroup$ Are your predictions the variable ZL? Or are you talking about the correct_prediction variable? What do you want to put in a DataFrame and what type is it? You can run type(my_variable) on it. $\endgroup$
    – n1k31t4
    Sep 5 '18 at 14:51

Thanks to Ankit Seth I found the answer.

#this returns output layer of forward prop with trained parameters of the model.
pred =  forward_propagation(X, parameters)
predictions_sigm = pred.eval(feed_dict = {X: X_test})
predictions_sigm = sigmoid(predictions)

#pred.eval returns linear regression layer for each class separately, so we need to pick index of maximum of them. I don't use softmax layer, since the output should be the same.

#init list of classes
y_list = []

for i in range(predictions_sigm.shape[1]):
    class_list = []

    #get index of maximum value of sigmoid. it will correspond to class
    y = np.argmax(class_list)

   #then append y_list to dataframe
   df['predicted_y'] = y_list

Create a list of the inputs, run each input through your model and save the prediction into a list then you can run the following code.

result = pd.DataFrame(data={'Id': YOUR_TEST_DATAFRAME['Id'], 'PREDICTION_COLUM_NAME': preds})
result.to_csv(path_or_buf='submittion.csv', index = False, header = True)

Then extract the prediction from a tensor in Tensorflow. This will extract data from Tensor:

pred = forward_propagation(X, parameters) 
predictions = pred.eval(feed_dict = {X: X_test})
  • 1
    $\begingroup$ you copied my post from Kaggle) $\endgroup$
    – Myron
    Sep 6 '18 at 10:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.