2
$\begingroup$

I'm using the following code to try and learn tensorflow. I've clearly specified the shapes of the training and validation X and y arrays.

import numpy as np
import tensorflow as tf

f = lambda x: 2*x
Xtrain = np.random.rand(400,1)
ytrain = f(Xtrain)
Xval = np.random.rand(200,1)
yval = f(Xval)

model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(10, activation='relu')
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.MeanSquaredError()
             )

model.fit(Xtrain, ytrain, epochs=50, verbose=0)

When I run yval.shape, model.predict(Xval).shape, I get the output ((200, 1), (200, 10)). I'm not able to understand where these extra 9 dimensions are coming from. Even the Xval prediction should have the shape (200, 1).

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The 10 outputs came from the fact that you have 10 neurons in the final layer of your network.

If you change your model to

model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(1, activation='relu')
])

its output will have a shape of (200, 1).

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you! Another thing - when I re-run this code, model.predict(Xval) gives an all-zero array. What am I missing? $\endgroup$
    – user9343456
    Jul 14, 2020 at 14:47
  • $\begingroup$ I just ran the code and it outputs non-zero values... Maybe you substituted an array with np.zeros() ? $\endgroup$
    – Djib2011
    Jul 14, 2020 at 15:13
  • $\begingroup$ Nope, no substitution. The code starting from Xtrain till the end is in a single Jupyter cell. When I run it the first time, the prediction is fine. But if I re-run, it gives all zeroes. Then I run for for a couple of times - again all zeroes. And after a few retries it gives a proper prediction again. Even if I put the randomized part in a prior cell, and run the code AFTER that in a separate cell repeatedly, the same thing happens. Something's off (I've changed the layer structure as you prescribed) $\endgroup$
    – user9343456
    Jul 14, 2020 at 15:19
  • $\begingroup$ Neural Networks are stochastic to a large degree. Maybe for some weight initializations the model outputs only zeros... I'm not sure. $\endgroup$
    – Djib2011
    Jul 14, 2020 at 15:38
  • 1
    $\begingroup$ I looked it up - turns out it may be due to the relu activation function. It can be replaced by tanh or leaky relu $\endgroup$
    – user9343456
    Jul 14, 2020 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.