Random accuracy results on validation set (Binary classification)

Question

I am configuring a CNN model for a classification problem of human action in Python using TensorFlow.

My data is video frames representing human body joints. Every 3 consecutive columns represent x,y,z coordinates of one joint.

I set the placeholders in tf to be of shape (None, l, j*3) where l=fixed number of frames and j= number of joints. Now the model is giving me very random results (pretty much noise) on validation.

• This is the code I used to compute loss and validation and plot results.

for e in range epochs:

# Loop over batches
        for x,y in random_batches(X_train, Y_train, batch_size):

            # Feed dictionary
            feed = {X : x, Y : y, keep_prob_ : 0.5, learning_rate_ : learning_rate}

            # Loss
            loss, _ , acc = sess.run([cost, optimizer, accuracy], feed_dict = feed)
            train_acc.append(acc)
            train_loss.append(loss)

            # Print acc & loss every 5 iter.s

            if (iteration % 5 == 0):
                print("Epoch: {}/{}".format(e, epochs), "Train acc: {:.6f}".format(acc))


            # Compute validation loss every 10 iter.s

            if (iteration%10 == 0):                
                val_acc_ = []
                val_loss_ = []

                for x_v, y_v in random_batches(X_valid, Y_valid, batch_size):
                    # Feed
                    feed = {X : x_v, Y : y_v, keep_prob_ : 1.0}  

                    # Loss
                    loss_v, acc_v = sess.run([cost, accuracy], feed_dict = feed)                    
                    val_acc_.append(acc_v)
                    val_loss_.append(loss_v)

                # Print
                print("Epoch: {}/{}".format(e, epochs), "Validation acc: {:.6f}".format(np.mean(val_acc_)))

                # Store
                validation_acc.append(np.mean(val_acc_))
                validation_loss.append(np.mean(val_loss_))


            # Iterate 
            iteration += 1


    ## Plotting:

t = np.arange(iteration-1)

plt.plot(t, np.array(train_loss), 'r-', t[t % 10 == 0], np.array(validation_loss), 'b*')

• So I thought maybe if I tell the system to consider that every 3 consecutive columns are representing just one joint it would solve this randomness on validation.

Not sure how to do this as tensorflow accepts arrays. And arrays don't have headers unlike dataframes.

0  0.30467  0.45957 -0.95414  1.74687  1.42338 -0.03860   
1  0.27331  0.59293 -1.00874  1.74135  1.32004 -0.00701       
2  0.30348  0.88129 -1.05517  1.75090  1.65138 -0.03112      

What I would want the system to understand is that the first 3 columns represent the same joint and the 2nd 3 columns represent the 2nd joint.

I tried shaping the data to be (None, l, j, 3) but that also gave me random results on validation as well. And when I looked up similar works, they were feeding tf data such as (None, l, j*3).

• Another area where I am doubting is wrong (although less likely to be wrong) is the shuffling of the data. I have my inputs data (X) and I have constructed the labels (Y) from the filenames. Below is a snippet of the get batches function (which does the shuffling):

# 1- Shuffle (X, Y)
m= X.shape[0]

    permutation = list(np.random.permutation(m))
    shuffled_X = X[permutation,:,:]
    shuffled_Y = Y[permutation,:]


    # 2- Partition (shuffled_X, shuffled_Y). Minus the end case.

    num_batches = m // batch_size # number of mini batches of size batch_size in the partitionning        
    X, Y = shuffled_X[:num_batches*batch_size], shuffled_Y[:num_batches*batch_size]

    # Handling the end case (last mini-batch < mini_batch_size)
    if m % batch_size != 0:
        X = shuffled_X[num_batches * batch_size : m,:,:]
        Y = shuffled_Y[num_batches * batch_size : m,:]

    # Loop over batches and yield
    for b in range(0, len(X), batch_size):
        yield X[b:b+batch_size], Y[b:b+batch_size]

I am a beginner in deep learning and tensorflow. Appreciate any advice or help here.

Answer 1

Ah, I think I've found the problem with the discrete values at least. In your random batching, you overwrite X and Y to be the remainder of the dataset here:

if m % batch_size != 0:
    X = shuffled_X[num_batches * batch_size : m,:,:]
    Y = shuffled_Y[num_batches * batch_size : m,:]

These datasets only have size $200-3*64=8$.

It's surprising that the training accuracies work out nicer then; is the training dataset perhaps a multiple of 64?

(After you get this fixed up, your original question is still interesting; maybe it should be posted as a separate question?)