2
$\begingroup$

I'm a beginner with ANN and DL in general. I have a regression task with a target of 2-dimensions, my dataset only have 46 samples (small dataset I think). I tried the code below that does a regression with only one output which works normally.

When I change to a two dimensional regression, my loss function becomes equal to NaN. I tried to change the optimizer and fix the dropout rate, but nothing changed, any solution?

import numpy
import pandas
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

#load data
dataframe = pandas.read_excel("data1.xlsx")
dataframe.isnull().any()

dataset = dataframe.values

X = dataset[:, 0:5]
Y = dataset[:,5:7]

def baseline_model():
    #create model
    model = Sequential()
    model.add(Dense(10, input_dim=5, kernel_initializer='normal', activation 
='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(2, kernel_initializer='normal'))

    #compile model
    #model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')
    model.compile(loss='mean_squared_error', optimizer='adam')
    return model



#fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)

estimators=[]
estimators.append(('standardize', StandardScaler()))
estimators.append(('mlp', KerasRegressor(build_fn=baseline_model, epochs=50, batch_size=32, verbose=0)))
pipeline = Pipeline(estimators)

kfold = KFold(n_splits=10, random_state = seed)
results = cross_val_score(pipeline, X, Y, cv=kfold)

print ("Result: %.2f (%.2f) MSE" %(results.mean(), results.std()))
$\endgroup$
  • $\begingroup$ NaN are sometimes due to too large learning rate. Try to set it 10 times lower, or so. $\endgroup$ – Dmytro Prylipko Jan 21 '19 at 15:19
  • $\begingroup$ I think your dataset is way too small. I guess your model have a big bias error and you are additionally using regurlarization, dropping 50% of neurons after the first hidden layer. Can you share your dataset? $\endgroup$ – MaxU Jan 21 '19 at 17:29
1
$\begingroup$

I faced same issues and searched solution in many places. But apparently, it's because I put wrong label in my dataset. It should be 0,1,2,... n but I made 1,2,3,....n. It should be started from zero. After I changed the label, it works well now.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.