# Why I am having odd results with this neural network model

I am kind of new to neural networks, so I was doing some practice with iris data set. But the results are weird. Because I am running the network with the same data (not randomly chosen) and huge number of epochs (500 times), but results are sometimes 0,25 sometimes 1,0 or 0,95. The only thing that random is biases and weights. I think training the network with 500 epochs should remove this randomness at the end. But it seems like it doesn't or there is some another problem that I am not noticed yet. If you help me about this I will be thankful.

(PS: If you give me an advice about making the question better I will edit it, I am new at forum)

import tensorflow as tf
import pandas as pd
import numpy as np
from tqdm import tqdm

def to_one_hot(label_data):
one_hot = []
for item in label_data:
temp = []
for i in range(3):
if(i == item):
temp.append(1)
else:
temp.append(0)
one_hot.append(temp)
return one_hot

x = []
y = []
species_dict = {'Iris-setosa' : 0,
'Iris-versicolor' : 1,
'Iris-virginica' : 2}
for i in range(len(df_data['Id'])):
temp_item = []
temp_item.append(df_data['SepalLengthCm'][i])
temp_item.append(df_data['SepalWidthCm'][i])
temp_item.append(df_data['PetalLengthCm'][i])
temp_item.append(df_data['PetalWidthCm'][i])
x.append(temp_item)
y.append(species_dict[df_data['Species'][i]])
y = to_one_hot(y)
x_train = []
y_train = []
x_test = []
y_test = []
for i in range(50):
if (i < n):
for t in range(3):
x_train.append(x[i+50*t])
y_train.append(y[i+50*t])
else:
for t in range(3):
x_test.append(x[i+50*t])
y_test.append(y[i+50*t])
return x_train,x_test,y_train,y_test

n_layers = 30
n_nodes = 7
n_classes = 3

x = tf.placeholder('float', [None,4])
y = tf.placeholder('float')

def neural_network_model(data):
layers = []
layers.append({'weights':tf.Variable(tf.random_normal([4,n_nodes])),
'biases':tf.Variable(tf.random_normal([n_nodes]))})
for i in range(n_layers-1):
layers.append({'weights':tf.Variable(tf.random_normal([n_nodes,n_nodes])),
'biases':tf.Variable(tf.random_normal([n_nodes]))})

output_layer = {'weights':tf.Variable(tf.random_normal([n_nodes,n_classes])),
'biases':tf.Variable(tf.random_normal([n_classes]))}

for i in range(n_layers):
if (i == 0):
nl = tf.nn.relu(nl)
else:
nl = tf.nn.relu(nl)

return output

def train_neural_network(x):
prediction = neural_network_model(x)
cost = tf.reduce_mean( tf.nn.softmax_cross_entropy_with_logits(logits = prediction, labels = y) )

hm_epochs = 500

with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
for epoch in tqdm(range(hm_epochs)):
_ ,c = sess.run([optimizer,cost], feed_dict = {x:x_train, y:y_train})
correct = tf.equal(tf.argmax(prediction,1),tf.argmax(y,1))
accuracy = tf.reduce_mean(tf.cast(correct, 'float'))
print('Accuracy : ',accuracy.eval({x:x_test,y:y_test}))

train_neural_network(x)

• Try setting up some seeds first? – Aditya Aug 2 at 4:16

Neural network algorithms are stochastic.

This means they make use of randomness, such as initializing to random weights, and in turn the same network trained on the same data can produce different results.The random initialization allows the network to learn a good approximation for the function being learned. The most common form of randomness used in neural networks is the random initialization of the network weights. Although randomness can be used in other areas, here is just a short list:

• Randomness in Initialization, such as weights.
• Randomness in Regularization, such as dropout
• Randomness in Layers, such as word embedding.
• Randomness in Optimization, such as stochastic optimization.

The Solutions.
The are two main solutions.

from numpy.random import seed