1
$\begingroup$

I am currently trying to predict a function that has a shape similar to that of a normal distribution. It is defined as:

(4 γΩ )/((γ+2nγ)^2+(4Δ^2+2Ω^2))

I have tried to use relu, sigmoid, and tanh activation functions.

I have also tried mae, mse and binary_crossentropy loss functions.

I have also tried adam, rmsprop and sgd optimizers, and have also played around with learning rates.

Nothing seems to work. When I perform a regression analysis, I'm told that the error percentage, mse and rmse values are low.

But when I plot the predictions, it just shows a linear function.

Any suggestions?

My architecture looks like:

df=pd.read_csv('DataGen(N=1,Gamma=0.1,Omega=5).csv')
df.head()
dataset=df.values
X=dataset[:1800000,0].reshape(-1,1)
Y=dataset[:1800000,2].reshape(-1,1)
X2=dataset[1800001:,0].reshape(-1,1)
Y2=dataset[1800001:,2].reshape(-1,1)
scaler= MinMaxScaler(feature_range=(0,1))
X_min=scaler.fit_transform(X)
Y_min=scaler.fit_transform(Y.reshape(-1,1))

X_test=scaler.fit_transform(X2)
Y_test=scaler.fit_transform(Y2.reshape(-1,1))

seed=7
np.random.seed(seed)
X_tr,X_val, Y_tr, Y_val = train_test_split(X,Y,test_size=0.1, random_state=seed)

def NN():
model= Sequential()
model.add(Dense(4, input_dim=4, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(16, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(32, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(64, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(128, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(64, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(32, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(16, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(3, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(1,kernel_initializer = 'normal'))

model.compile(loss='mean_absolute_error', optimizer='adam', metrics=['MAE'])
return model

My plotted results look kind of like this:

Where the dependent variable is the prediction and the dependent variable is delta.

The blue points are the expected values and the red points are predictions.

Improve this question
$\endgroup$
2
  • $\begingroup$ What is your model architecture? Can we see some examples of inputs to your function and its respective output? $\endgroup$
    – JahKnows
    Sep 8, 2018 at 19:18
  • $\begingroup$ Please recheck the question, I redited it now. Thanks $\endgroup$
    – DeepLearner
    Sep 8, 2018 at 20:20

1 Answer 1

Reset to default
2
$\begingroup$

My guess is that the problem is with your first layer: model.add(Dense(1, input_dim=1, activation='tanh',kernel_initializer = 'normal')). With a single hidden unit, a lot of information from the input layer will not be available to subsequent layers.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Sorry, I copied the wrong code. You are correct, the first layer should have 4 input dimensions. But I still had this problem. $\endgroup$
    – DeepLearner
    Sep 8, 2018 at 20:51
  • $\begingroup$ I think even 4 might be too small. Is there a reason you have the number of neurons starting small and then increasing with each layer to 128 neurons, and then decreasing with each layer? $\endgroup$
    – from keras import michael
    Sep 8, 2018 at 20:53
  • $\begingroup$ Not really. I just started with 4 because I have 4 input variables...I wasn't sure if more would be better. Is there any particular architecture you wod reccomend? $\endgroup$
    – DeepLearner
    Sep 8, 2018 at 20:59
  • 1
    $\begingroup$ If it were me, I'd start with one dense layer with 32 neurons. model= Sequential() model.add(Dense(32, input_dim=4, activation='relu') model.add(Dense(1, activation = None)) Then, I'd see if more or less neurons makes any difference. Then I would add another layer with 32 neurons, and repeat until I was happy with the result. A network with many layers is typically used with hierarchical relationships; with 4 inputs, I don't think you should require more than one or two layers. $\endgroup$
    – from keras import michael
    Sep 8, 2018 at 21:13
  • $\begingroup$ Are there any particular loss functions or optimizers you think would work best? $\endgroup$
    – DeepLearner
    Sep 9, 2018 at 12:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.