# Keras model.predict giving different shape from training label array

I'm using the following code to try and learn tensorflow. I've clearly specified the shapes of the training and validation X and y arrays.

import numpy as np
import tensorflow as tf

f = lambda x: 2*x
Xtrain = np.random.rand(400,1)
ytrain = f(Xtrain)
Xval = np.random.rand(200,1)
yval = f(Xval)

model = tf.keras.models.Sequential([
tf.keras.layers.Dense(10, activation='relu'),
tf.keras.layers.Dense(10, activation='relu'),
tf.keras.layers.Dense(10, activation='relu'),
tf.keras.layers.Dense(10, activation='relu')
])

loss=tf.keras.losses.MeanSquaredError()
)

model.fit(Xtrain, ytrain, epochs=50, verbose=0)


When I run yval.shape, model.predict(Xval).shape, I get the output ((200, 1), (200, 10)). I'm not able to understand where these extra 9 dimensions are coming from. Even the Xval prediction should have the shape (200, 1).

The 10 outputs came from the fact that you have 10 neurons in the final layer of your network.

If you change your model to

model = tf.keras.models.Sequential([
tf.keras.layers.Dense(10, activation='relu'),
tf.keras.layers.Dense(10, activation='relu'),
tf.keras.layers.Dense(10, activation='relu'),
tf.keras.layers.Dense(1, activation='relu')
])


its output will have a shape of (200, 1).

• Thank you! Another thing - when I re-run this code, model.predict(Xval) gives an all-zero array. What am I missing? Jul 14, 2020 at 14:47
• I just ran the code and it outputs non-zero values... Maybe you substituted an array with np.zeros() ? Jul 14, 2020 at 15:13
• Nope, no substitution. The code starting from Xtrain till the end is in a single Jupyter cell. When I run it the first time, the prediction is fine. But if I re-run, it gives all zeroes. Then I run for for a couple of times - again all zeroes. And after a few retries it gives a proper prediction again. Even if I put the randomized part in a prior cell, and run the code AFTER that in a separate cell repeatedly, the same thing happens. Something's off (I've changed the layer structure as you prescribed) Jul 14, 2020 at 15:19
• Neural Networks are stochastic to a large degree. Maybe for some weight initializations the model outputs only zeros... I'm not sure. Jul 14, 2020 at 15:38
• I looked it up - turns out it may be due to the relu activation function. It can be replaced by tanh or leaky relu Jul 14, 2020 at 15:39