I am configuring a CNN model for a classification problem of human action in Python using TensorFlow.

My data is video frames representing human body joints. Every 3 consecutive columns represent x,y,z coordinates of one joint.

I set the placeholders in tf to be of shape (None, l, j*3) where l=fixed number of frames and j= number of joints. Now the model is giving me very random results (pretty much noise) on validation.

  • This is the code I used to compute loss and validation and plot results.
for e in range epochs:

# Loop over batches
        for x,y in random_batches(X_train, Y_train, batch_size):

            # Feed dictionary
            feed = {X : x, Y : y, keep_prob_ : 0.5, learning_rate_ : learning_rate}

            # Loss
            loss, _ , acc = sess.run([cost, optimizer, accuracy], feed_dict = feed)

            # Print acc & loss every 5 iter.s

            if (iteration % 5 == 0):
                print("Epoch: {}/{}".format(e, epochs), "Train acc: {:.6f}".format(acc))

            # Compute validation loss every 10 iter.s

            if (iteration%10 == 0):                
                val_acc_ = []
                val_loss_ = []

                for x_v, y_v in random_batches(X_valid, Y_valid, batch_size):
                    # Feed
                    feed = {X : x_v, Y : y_v, keep_prob_ : 1.0}  

                    # Loss
                    loss_v, acc_v = sess.run([cost, accuracy], feed_dict = feed)                    

                # Print
                print("Epoch: {}/{}".format(e, epochs), "Validation acc: {:.6f}".format(np.mean(val_acc_)))

                # Store

            # Iterate 
            iteration += 1

    ## Plotting:

t = np.arange(iteration-1)

plt.plot(t, np.array(train_loss), 'r-', t[t % 10 == 0], np.array(validation_loss), 'b*')

  • So I thought maybe if I tell the system to consider that every 3 consecutive columns are representing just one joint it would solve this randomness on validation.

Not sure how to do this as tensorflow accepts arrays. And arrays don't have headers unlike dataframes.

0  0.30467  0.45957 -0.95414  1.74687  1.42338 -0.03860   
1  0.27331  0.59293 -1.00874  1.74135  1.32004 -0.00701       
2  0.30348  0.88129 -1.05517  1.75090  1.65138 -0.03112      

What I would want the system to understand is that the first 3 columns represent the same joint and the 2nd 3 columns represent the 2nd joint.

I tried shaping the data to be (None, l, j, 3) but that also gave me random results on validation as well. And when I looked up similar works, they were feeding tf data such as (None, l, j*3).

  • Another area where I am doubting is wrong (although less likely to be wrong) is the shuffling of the data. I have my inputs data (X) and I have constructed the labels (Y) from the filenames. Below is a snippet of the get batches function (which does the shuffling):
# 1- Shuffle (X, Y)
m= X.shape[0]

    permutation = list(np.random.permutation(m))
    shuffled_X = X[permutation,:,:]
    shuffled_Y = Y[permutation,:]

    # 2- Partition (shuffled_X, shuffled_Y). Minus the end case.

    num_batches = m // batch_size # number of mini batches of size batch_size in the partitionning        
    X, Y = shuffled_X[:num_batches*batch_size], shuffled_Y[:num_batches*batch_size]

    # Handling the end case (last mini-batch < mini_batch_size)
    if m % batch_size != 0:
        X = shuffled_X[num_batches * batch_size : m,:,:]
        Y = shuffled_Y[num_batches * batch_size : m,:]

    # Loop over batches and yield
    for b in range(0, len(X), batch_size):
        yield X[b:b+batch_size], Y[b:b+batch_size]

I am a beginner in deep learning and tensorflow. Appreciate any advice or help here.

  • $\begingroup$ Well, ignoring everything else for now, the apparently discrete accuracy values suggests you only have...8(?) validation points, which seems to small to be meaningful. $\endgroup$
    – Ben Reiniger
    Aug 31, 2019 at 13:34
  • $\begingroup$ @BenReiniger Thanks for the comment. I will try and increase the size of the validation. I had segmented the data as: 80% test, 10% validation and 10% test. $\endgroup$
    – R.A
    Aug 31, 2019 at 13:58
  • $\begingroup$ I am not sure why this 8 discrete points are there indeed. When I tuned the hyperparameters again, I just noticed, again the same 8 discrete values occur. Not sure why it is 8 because I have 200 files for validation. $\endgroup$
    – R.A
    Aug 31, 2019 at 14:04
  • $\begingroup$ Could those scores just be for individual batches? $\endgroup$
    – Ben Reiniger
    Aug 31, 2019 at 14:06
  • $\begingroup$ I'm not sure. Basically, I'm saving train accuracy every 5 batches, plotting and computing validation mean at every 10 iterations, batch size=64. (NB: having 200 validation files, i would have only 3.125 batches in every iteration). $\endgroup$
    – R.A
    Aug 31, 2019 at 14:57

1 Answer 1


Ah, I think I've found the problem with the discrete values at least. In your random batching, you overwrite X and Y to be the remainder of the dataset here:

if m % batch_size != 0:
    X = shuffled_X[num_batches * batch_size : m,:,:]
    Y = shuffled_Y[num_batches * batch_size : m,:]

These datasets only have size $200-3*64=8$.

It's surprising that the training accuracies work out nicer then; is the training dataset perhaps a multiple of 64?

(After you get this fixed up, your original question is still interesting; maybe it should be posted as a separate question?)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.