Neural network is not giving the expected output after training in Python

Question

My neural network is not giving the expected output after training in Python. Is there any error in the code? Is there any way to reduce the mean squared error (MSE)?

I tried to train (Run the program) the network repeatedly but it is not learning, instead it is giving the same MSE and output.

Here is the Data I used:

https://drive.google.com/open?id=1GLm87-5E_6YhUIPZ_CtQLV9F9wcGaTj2

Here is my code:

#load and evaluate a saved model
from numpy import loadtxt
from tensorflow.keras.models import load_model

# load model
model = load_model('ANNnew.h5')
# summarize model.
model.summary()
#Model starts
import numpy as np
import pandas as pd 
from tensorflow.keras.layers import Dense, Activation
from tensorflow.keras.models import Sequential
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
# Importing the dataset
X = pd.read_excel(r"C:\filelocation\Data.xlsx","Sheet1").values
y = pd.read_excel(r"C:\filelocation\Data.xlsx","Sheet2").values

# Splitting the dataset into the Training set and Test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.08, random_state = 0)

# Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

# Initialising the ANN
model = Sequential()

# Adding the input layer and the first hidden layer
model.add(Dense(32, activation = 'tanh', input_dim = 4))

# Adding the second hidden layer
model.add(Dense(units = 18, activation = 'tanh'))

# Adding the third hidden layer
model.add(Dense(units = 32, activation = 'tanh'))

#model.add(Dense(1))
model.add(Dense(units = 1))

# Compiling the ANN
model.compile(optimizer = 'adam', loss = 'mean_squared_error')

# Fitting the ANN to the Training set
model.fit(X_train, y_train, batch_size = 100, epochs = 1000)

y_pred = model.predict(X_test)
for i in range(5):
    print('%s => %d (expected %s)' % (X[i].tolist(), y_pred[i], y[i].tolist()))


plt.plot(y_test, color = 'red', label = 'Test data')
plt.plot(y_pred, color = 'blue', label = 'Predicted data')
plt.title('Prediction')
plt.legend()
plt.show()

# save model and architecture to single file
model.save("ANNnew.h5")
print("Saved model to disk")

Answer 1

It looks like your loss is improving (the loss is decreasing every epoch), so I wouldn't say the network isn't training. Try looking at a training plot (which would have epoch on the x-axis and loss on the y-axis). See if there are any jumps or strange patterns in the training. If you don't see a roughly decreasing training loss, then you have a problem.

It's also possible that this model is just completely overfitting your data, leading to poor prediction on the test set. How many inputs do you have? Why are you using three hidden layers? Try reducing the size of your model and see if your prediction improves.

Answer 2

From what I can see here, your model has a lot of scope for overfitting. You're training this model with only about 70 data points. Which is less in my opinion. You could try to either increase your training data and then observe the predictions and see if they have improved, or you could reduce the size of your model and build one with fewer parameters.