1
$\begingroup$

I am currently trying to predict a function that has a shape similar to that of a normal distribution. It is defined as:

(4 γΩ )/((γ+2nγ)^2+(4Δ^2+2Ω^2))

I have tried to use relu, sigmoid, and tanh activation functions.

I have also tried mae, mse and binary_crossentropy loss functions.

I have also tried adam, rmsprop and sgd optimizers, and have also played around with learning rates.

Nothing seems to work. When I perform a regression analysis, I'm told that the error percentage, mse and rmse values are low.

But when I plot the predictions, it just shows a linear function.

Any suggestions?

My architecture looks like:

df=pd.read_csv('DataGen(N=1,Gamma=0.1,Omega=5).csv')
df.head()
dataset=df.values
X=dataset[:1800000,0].reshape(-1,1)
Y=dataset[:1800000,2].reshape(-1,1)
X2=dataset[1800001:,0].reshape(-1,1)
Y2=dataset[1800001:,2].reshape(-1,1)
scaler= MinMaxScaler(feature_range=(0,1))
X_min=scaler.fit_transform(X)
Y_min=scaler.fit_transform(Y.reshape(-1,1))

X_test=scaler.fit_transform(X2)
Y_test=scaler.fit_transform(Y2.reshape(-1,1))

seed=7
np.random.seed(seed)
X_tr,X_val, Y_tr, Y_val = train_test_split(X,Y,test_size=0.1, random_state=seed)

def NN():
model= Sequential()
model.add(Dense(4, input_dim=4, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(16, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(32, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(64, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(128, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(64, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(32, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(16, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(3, activation='tanh',kernel_initializer = 'normal'))
model.add(BatchNormalization())
model.add(Dense(1,kernel_initializer = 'normal'))

model.compile(loss='mean_absolute_error', optimizer='adam', metrics=['MAE'])
return model

My plotted results look kind of like this:

Where the dependent variable is the prediction and the dependent variable is delta.

The blue points are the expected values and the red points are predictions.

$\endgroup$
  • $\begingroup$ What is your model architecture? Can we see some examples of inputs to your function and its respective output? $\endgroup$ – JahKnows Sep 8 '18 at 19:18
  • $\begingroup$ Please recheck the question, I redited it now. Thanks $\endgroup$ – DeepLearner Sep 8 '18 at 20:20
2
$\begingroup$

My guess is that the problem is with your first layer: model.add(Dense(1, input_dim=1, activation='tanh',kernel_initializer = 'normal')). With a single hidden unit, a lot of information from the input layer will not be available to subsequent layers.

$\endgroup$
  • $\begingroup$ Sorry, I copied the wrong code. You are correct, the first layer should have 4 input dimensions. But I still had this problem. $\endgroup$ – DeepLearner Sep 8 '18 at 20:51
  • $\begingroup$ I think even 4 might be too small. Is there a reason you have the number of neurons starting small and then increasing with each layer to 128 neurons, and then decreasing with each layer? $\endgroup$ – from keras import michael Sep 8 '18 at 20:53
  • $\begingroup$ Not really. I just started with 4 because I have 4 input variables...I wasn't sure if more would be better. Is there any particular architecture you wod reccomend? $\endgroup$ – DeepLearner Sep 8 '18 at 20:59
  • 1
    $\begingroup$ If it were me, I'd start with one dense layer with 32 neurons. model= Sequential() model.add(Dense(32, input_dim=4, activation='relu') model.add(Dense(1, activation = None)) Then, I'd see if more or less neurons makes any difference. Then I would add another layer with 32 neurons, and repeat until I was happy with the result. A network with many layers is typically used with hierarchical relationships; with 4 inputs, I don't think you should require more than one or two layers. $\endgroup$ – from keras import michael Sep 8 '18 at 21:13
  • $\begingroup$ Are there any particular loss functions or optimizers you think would work best? $\endgroup$ – DeepLearner Sep 9 '18 at 12:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.