0
$\begingroup$

I am trying to make a simple neural network with one dependent and one independent variable. Could you maybe give me a tutorial or help me with the implementation of a neural network with one dependent and one independent variable. So far I have the following code, however my predictions are not good although the error is minimized. Should I scale X and Y or do I have some mistake?

Thank you in advance

import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np

x=[(i*i)+0.2 for i in range(1000)]
y=[i for i in range(1000)]

x_train=np.reshape(x,(-1,1))
y_train=np.reshape(y,(-1,1))
x_test=x_train[:,-10:]
y_test=y_train[:,-10:]
plt.scatter(x_train,y_train)
plt.show()


X=tf.placeholder(tf.float32,[None,1])
Y=tf.placeholder(tf.float32,[None,1])

n_inputs=1
n_hidden_1=20
n_hidden_2=20
n_outputs=1

weights={
    "h1": tf.Variable(tf.random_normal([n_inputs,n_hidden_1])),
    "h2": tf.Variable(tf.random_normal([n_hidden_1,n_hidden_2])),
    "out": tf.Variable(tf.random_normal([n_hidden_2,n_outputs]))
}

biases={
    "b1": tf.Variable(tf.random_normal([n_hidden_1])),
    "b2": tf.Variable(tf.random_normal([n_hidden_2])),
    "out": tf.Variable(tf.random_normal([n_outputs]))
}

def neural_net(x):
    layer_1=tf.add(tf.matmul(x,weights["h1"]),biases["b1"])
    layer_1=tf.nn.relu(layer_1)
    layer_2=tf.add(tf.matmul(layer_1,weights["h2"]),biases["b2"])
    layer_2=tf.nn.relu(layer_2)
    layer_3=tf.matmul(layer_2,weights["out"])+biases["out"]
    return layer_3

Y_pred=neural_net(X)

loss=tf.losses.mean_squared_error(X,Y_pred)
optimizer=tf.train.AdamOptimizer(learning_rate=0.01)
train_op=optimizer.minimize(loss)

epochs=1000
init=tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)
    for i in range(epochs):
        sess.run(train_op,feed_dict={X:x_train,Y:y_train})
        loss_op=sess.run(loss,feed_dict={X:x_train,Y:y_train})
        if i%10==0:
            print("Epoch "+str(i)+" loss "+str(loss_op))
    pred=sess.run(Y_pred,feed_dict={X:x_test})
    plt.plot(pred,color="red")
    plt.plot(y_test,color="blue")
    plt.show()
    plt.scatter(pred,y_test)
    plt.show()
    for i in range(len(pred)):
        print(str(pred[i])+" "+str(y_test[i]))
$\endgroup$
  • $\begingroup$ If you just want to make a simple neural network with one dependent and one independent variable, I would suggest to use scikit-learn instead of tensorflow. $\endgroup$ – Ankit Seth Jul 11 '19 at 11:08
  • $\begingroup$ I would ideally extend it to a recursive neural network but I wanted to start with something. $\endgroup$ – Ana Smile Jul 11 '19 at 11:19
  • 1
    $\begingroup$ Try shuffling your data before train test split. One reason might be that you are using bigger values in test than in train. $\endgroup$ – Ankit Seth Jul 11 '19 at 12:00
0
$\begingroup$

Your predictions are not actually that bad. At the very last line of your code, print the expected value too at each line (that is x_test[i]= y_test[i]^2+0.2).

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Shouldn't y_test[i] correspond to pred[i]? $\endgroup$ – Ana Smile Jul 11 '19 at 13:34
  • $\begingroup$ pred[i] is what your model predicts. You should compare it to the actual results which is calculated as y_test[i]^2+0.2 according to your code. $\endgroup$ – serali Jul 11 '19 at 14:08
  • $\begingroup$ or x_test[i] to be clear. That is how x and y are defined. $\endgroup$ – serali Jul 11 '19 at 14:26
  • $\begingroup$ One more question, pred[i] should correspond to y_test[i] so my intuition was to compare with that. I want the model to output y_test[i] not x_test[i] or y_test[i]^2+0.2. Since this x_test and y_test will not be defined as is, but will be other numbers, why does the model not output y_test directly? Do I need to normalize or scale them somehow? $\endgroup$ – Ana Smile Jul 12 '19 at 7:13
  • $\begingroup$ No, it is just related to how things are defined in the given code. We have on lines 5 to 11: x=[(i*i)+0.2 for i in range(1000)] y=[i for i in range(1000)] x_train=np.reshape(x,(-1,1)) y_train=np.reshape(y,(-1,1)) x_test=x_train[:,-10:] y_test=y_train[:,-10:] given above. These mean that y is the input and x is the output (output of the calculation y^2 +0.2). $\endgroup$ – serali Jul 12 '19 at 10:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.