import tensorflow as tf
x = tf.placeholder(tf.float32, [None,4]) # input vector
w1 = tf.Variable(tf.random_normal([4,2])) # weights between first and second layers
b1 = tf.Variable(tf.zeros([2])) # biases added to hidden layer
w2 = tf.Variable(tf.random_normal([2,1])) # weights between second and third layer
b2 = tf.Variable(tf.zeros([1])) # biases added to third (output) layer
def feedForward(x,w,b): # function for forward propagation
Input = tf.add(tf.matmul(x,w), b)
Output = tf.sigmoid(Input)
return Output
Out1 = feedForward(x,w1,b1) # output of first layer
Out2 = feedForward(Out1,w2,b2) # output of second layer
MHat = 50*Out2 # final prediction is in the range (0,50)
M = tf.placeholder(tf.float32, [None,1]) # placeholder for actual (target value of marks)
J = tf.reduce_mean(tf.square(MHat - M)) # cost function -- mean square errors
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(J) # minimize J using Gradient Descent
sess = tf.InteractiveSession() # create interactive session
tf.global_variables_initializer().run() # initialize all weight and bias variables with specified values
xs = [[1,3,9,7],
[7,9,8,2], # x training data
[2,4,6,5]]
Ms = [[47],
[43], # M training data
[39]]
for _ in range(1000): # performing learning process on training data 1000 times
sess.run(train_step, feed_dict = {x:xs, M:Ms})
>>> print(sess.run(MHat, feed_dict = {x:[[1,15,9,7]]}))
[[50.]]
>>> print(sess.run(MHat, feed_dict = {x:[[3,8,1,2]]}))
[[50.]]
>>> print(sess.run(MHat, feed_dict = {x:[[6,7,10,9]]}))
[[50.]]
In this code, I am trying to predict the marks M obtained by a student in a test out of 50 given how many hours he/she slept, studied, used electronics and played the day before the test. These 4 features come under the input feature vector x.
To solve this regression problem, I am using a deep neural network with an input layer with 4 perceptrons (the input features), a hidden layer with two perceptrons and an output layer with one perceptron.
I have used sigmoid as the activation function. But, I am getting the exact same prediction([[50.0]]) for M for all possible input vectors I feed in.
Can someone please tell me what is wrong with the code above, and why I get the same result each time?