Random accuracy results on validation set (Binary classification)

I am configuring a CNN model for a classification problem of human action in Python using TensorFlow.

My data is video frames representing human body joints. Every 3 consecutive columns represent x,y,z coordinates of one joint.

I set the placeholders in tf to be of shape (None, l, j*3) where l=fixed number of frames and j= number of joints. Now the model is giving me very random results (pretty much noise) on validation.

• This is the code I used to compute loss and validation and plot results.
for e in range epochs:

# Loop over batches
for x,y in random_batches(X_train, Y_train, batch_size):

# Feed dictionary
feed = {X : x, Y : y, keep_prob_ : 0.5, learning_rate_ : learning_rate}

# Loss
loss, _ , acc = sess.run([cost, optimizer, accuracy], feed_dict = feed)
train_acc.append(acc)
train_loss.append(loss)

# Print acc & loss every 5 iter.s

if (iteration % 5 == 0):
print("Epoch: {}/{}".format(e, epochs), "Train acc: {:.6f}".format(acc))

# Compute validation loss every 10 iter.s

if (iteration%10 == 0):
val_acc_ = []
val_loss_ = []

for x_v, y_v in random_batches(X_valid, Y_valid, batch_size):
# Feed
feed = {X : x_v, Y : y_v, keep_prob_ : 1.0}

# Loss
loss_v, acc_v = sess.run([cost, accuracy], feed_dict = feed)
val_acc_.append(acc_v)
val_loss_.append(loss_v)

# Print
print("Epoch: {}/{}".format(e, epochs), "Validation acc: {:.6f}".format(np.mean(val_acc_)))

# Store
validation_acc.append(np.mean(val_acc_))
validation_loss.append(np.mean(val_loss_))

# Iterate
iteration += 1

## Plotting:

t = np.arange(iteration-1)

plt.plot(t, np.array(train_loss), 'r-', t[t % 10 == 0], np.array(validation_loss), 'b*')


• So I thought maybe if I tell the system to consider that every 3 consecutive columns are representing just one joint it would solve this randomness on validation.

Not sure how to do this as tensorflow accepts arrays. And arrays don't have headers unlike dataframes.

0  0.30467  0.45957 -0.95414  1.74687  1.42338 -0.03860
1  0.27331  0.59293 -1.00874  1.74135  1.32004 -0.00701
2  0.30348  0.88129 -1.05517  1.75090  1.65138 -0.03112


What I would want the system to understand is that the first 3 columns represent the same joint and the 2nd 3 columns represent the 2nd joint.

I tried shaping the data to be (None, l, j, 3) but that also gave me random results on validation as well. And when I looked up similar works, they were feeding tf data such as (None, l, j*3).

• Another area where I am doubting is wrong (although less likely to be wrong) is the shuffling of the data. I have my inputs data (X) and I have constructed the labels (Y) from the filenames. Below is a snippet of the get batches function (which does the shuffling):
# 1- Shuffle (X, Y)
m= X.shape[0]

permutation = list(np.random.permutation(m))
shuffled_X = X[permutation,:,:]
shuffled_Y = Y[permutation,:]

# 2- Partition (shuffled_X, shuffled_Y). Minus the end case.

num_batches = m // batch_size # number of mini batches of size batch_size in the partitionning
X, Y = shuffled_X[:num_batches*batch_size], shuffled_Y[:num_batches*batch_size]

# Handling the end case (last mini-batch < mini_batch_size)
if m % batch_size != 0:
X = shuffled_X[num_batches * batch_size : m,:,:]
Y = shuffled_Y[num_batches * batch_size : m,:]

# Loop over batches and yield
for b in range(0, len(X), batch_size):
yield X[b:b+batch_size], Y[b:b+batch_size]


I am a beginner in deep learning and tensorflow. Appreciate any advice or help here.

• Well, ignoring everything else for now, the apparently discrete accuracy values suggests you only have...8(?) validation points, which seems to small to be meaningful. Commented Aug 31, 2019 at 13:34
• @BenReiniger Thanks for the comment. I will try and increase the size of the validation. I had segmented the data as: 80% test, 10% validation and 10% test.
– R.A
Commented Aug 31, 2019 at 13:58
• I am not sure why this 8 discrete points are there indeed. When I tuned the hyperparameters again, I just noticed, again the same 8 discrete values occur. Not sure why it is 8 because I have 200 files for validation.
– R.A
Commented Aug 31, 2019 at 14:04
• Could those scores just be for individual batches? Commented Aug 31, 2019 at 14:06
• I'm not sure. Basically, I'm saving train accuracy every 5 batches, plotting and computing validation mean at every 10 iterations, batch size=64. (NB: having 200 validation files, i would have only 3.125 batches in every iteration).
– R.A
Commented Aug 31, 2019 at 14:57

Ah, I think I've found the problem with the discrete values at least. In your random batching, you overwrite X and Y to be the remainder of the dataset here:
if m % batch_size != 0:

These datasets only have size $$200-3*64=8$$.