# Regression with Neural Networks in Tensorflow problem

I have recently started learning Neural networks and Python. I am trying out linear regression for a dataset with 14 features and 1 outcome. I have divided the data into training and test data. I have experimented with many parameters (learning rate, nodes per layer, number of layers, number of steps and optimization algorithm) but my test errors are as high as 150%.

I have posted my code below along with the cost curve (cost vs epochs). Where am making a mistake and what should I change? Or can you suggest some other important checks?

import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf

# importing features and observations data for training and validation
training_filename_X = "training_set_X.csv"
training_filename_Y = "training_set_Y.csv"
test_filename_X = "test_set_X.csv"
test_filename_Y = "test_set_Y.csv"

# normalizing training data
training_features_stddev_arr = np.std(training_features, axis=0)
training_features_mean_arr = np.mean(training_features, axis=0)
normalized_training_features = (training_features-training_features_mean_arr)/training_features_stddev_arr

# normalizing validation data with training set mean and standard deviation
normalized_validation_features = (validation_features-training_features_mean_arr)/training_features_stddev_arr

# normalizing test data with training set mean and standard deviation
normalized_test_features = (test_features-training_features_mean_arr)/training_features_stddev_arr

# layer parameters
n_nodes_hl1 = 20
n_nodes_hl2 = 20
n_nodes_hl3 = 20
no_features = 14
learning_rate = 0.01
epochs = 200

cost_history = np.empty(shape=[1], dtype=float)

X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

# defining weights for each layer taken from a normal distribution with variance 2/n
hl1_weight = tf.Variable(tf.random_normal([no_features, n_nodes_hl1], stddev=np.sqrt(2/no_features)))
hl2_weight = tf.Variable(tf.random_normal([n_nodes_hl1, n_nodes_hl2], stddev=np.sqrt(2/n_nodes_hl1)))
hl3_weight = tf.Variable(tf.random_normal([n_nodes_hl2, n_nodes_hl3], stddev=np.sqrt(2/n_nodes_hl2)))
output_weight = tf.Variable(tf.random_normal([n_nodes_hl3, 1], stddev=np.sqrt(2/n_nodes_hl3)))

# defining biases for each layer
hl1_bias = tf.Variable(tf.random_uniform([n_nodes_hl1], -1.0, 1.0))
hl2_bias = tf.Variable(tf.random_uniform([n_nodes_hl2], -1.0, 1.0))
hl3_bias = tf.Variable(tf.random_uniform([n_nodes_hl3], -1.0, 1.0))
output_bias = tf.Variable(tf.random_uniform([1], -1.0, 1.0))

# defining activation functions for each layer
hl1 = tf.sigmoid(tf.matmul(X, hl1_weight) + hl1_bias)
hl2 = tf.sigmoid(tf.matmul(hl1, hl2_weight) + hl2_bias)
hl3 = tf.sigmoid(tf.matmul(hl2, hl3_weight) + hl3_bias)
output = tf.matmul(hl3, output_weight) + output_bias

# using mean squared error cost function
cost  = tf.reduce_mean(tf.square(output - Y))

init = tf.global_variables_initializer()

# running the network
with tf.Session() as sess:
sess.run(init)

for step in np.arange(epochs):
sess.run(optimizer, feed_dict={X:normalized_training_features, Y:training_observations})
print (sess.run(cost, feed_dict={X:normalized_training_features, Y:training_observations}))
cost_history = np.append(cost_history, sess.run(cost,feed_dict={X:normalized_training_features, Y:training_observations}))

pred_y = sess.run(output, feed_dict={X:normalized_test_features})
print (sess.run(output, feed_dict={X:normalized_test_features}))
mse = tf.reduce_mean(tf.square(pred_y - test_observations))
print("MSE: %4f" % sess.run(mse))

# plotting the cost history
plt.plot(range(len(cost_history)), cost_history)
plt.axis([0, epochs, 0, np.max(cost_history)])
plt.show()


• How many instances do you have in your dataset? And can you plot the predicted vs the actual? – JahKnows Jun 19 '18 at 2:07
• @JahKnows I have 900 instances in my data set and 169 in my test set. I have 14 features. I don't know if they are sufficient. What is your opinion? – user53799 Jun 20 '18 at 5:11