# Using Keras to Predict a Function Following a Normal Distribution

I am currently trying to predict a function that has a shape similar to that of a normal distribution. It is defined as:

(4 γΩ )/((γ+2nγ)^2+(4Δ^2+2Ω^2))


I have tried to use relu, sigmoid, and tanh activation functions.

I have also tried mae, mse and binary_crossentropy loss functions.

I have also tried adam, rmsprop and sgd optimizers, and have also played around with learning rates.

Nothing seems to work. When I perform a regression analysis, I'm told that the error percentage, mse and rmse values are low.

But when I plot the predictions, it just shows a linear function.

Any suggestions?

My architecture looks like:

df=pd.read_csv('DataGen(N=1,Gamma=0.1,Omega=5).csv')
df.head()
dataset=df.values
X=dataset[:1800000,0].reshape(-1,1)
Y=dataset[:1800000,2].reshape(-1,1)
X2=dataset[1800001:,0].reshape(-1,1)
Y2=dataset[1800001:,2].reshape(-1,1)
scaler= MinMaxScaler(feature_range=(0,1))
X_min=scaler.fit_transform(X)
Y_min=scaler.fit_transform(Y.reshape(-1,1))

X_test=scaler.fit_transform(X2)
Y_test=scaler.fit_transform(Y2.reshape(-1,1))

seed=7
np.random.seed(seed)
X_tr,X_val, Y_tr, Y_val = train_test_split(X,Y,test_size=0.1, random_state=seed)

def NN():
model= Sequential()
model.add(Dense(4, input_dim=4, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(16, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(32, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(64, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(128, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(64, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(32, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(16, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(3, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(1,kernel_initializer = 'normal'))

model.compile(loss='mean_absolute_error', optimizer='adam', metrics=['MAE'])
return model


My plotted results look kind of like this:

The blue points are the expected values and the red points are predictions.

• What is your model architecture? Can we see some examples of inputs to your function and its respective output? Sep 8, 2018 at 19:18
• Please recheck the question, I redited it now. Thanks Sep 8, 2018 at 20:20

## 1 Answer

My guess is that the problem is with your first layer: model.add(Dense(1, input_dim=1, activation='tanh',kernel_initializer = 'normal')). With a single hidden unit, a lot of information from the input layer will not be available to subsequent layers.

• Sorry, I copied the wrong code. You are correct, the first layer should have 4 input dimensions. But I still had this problem. Sep 8, 2018 at 20:51
• I think even 4 might be too small. Is there a reason you have the number of neurons starting small and then increasing with each layer to 128 neurons, and then decreasing with each layer? Sep 8, 2018 at 20:53
• Not really. I just started with 4 because I have 4 input variables...I wasn't sure if more would be better. Is there any particular architecture you wod reccomend? Sep 8, 2018 at 20:59
• If it were me, I'd start with one dense layer with 32 neurons. model= Sequential() model.add(Dense(32, input_dim=4, activation='relu') model.add(Dense(1, activation = None)) Then, I'd see if more or less neurons makes any difference. Then I would add another layer with 32 neurons, and repeat until I was happy with the result. A network with many layers is typically used with hierarchical relationships; with 4 inputs, I don't think you should require more than one or two layers. Sep 8, 2018 at 21:13
• Are there any particular loss functions or optimizers you think would work best? Sep 9, 2018 at 12:13