3
$\begingroup$

I've trained a model with Keras, saved it and when I'm trying to apply it on new data, I'm encountering an error :

ValueError: Error when checking : expected dense_1_input to have shape (None, 5) but got array with shape (200, 1)

Here's the code for training and saving the trained model :

# Import necessary modules
import numpy as np  # numpy is just used for reading the data
import keras
from keras.layers import Dense
from keras.models import Sequential
from keras.models import load_model # To save and load model


# fix random seed for reproducibility
seed = 7
np.random.seed(seed)

# load the dataset
dataset = np.loadtxt("modiftrain.csv", delimiter=";")

# split into input (X) and output (Y) variables
X = dataset[:,0:5]
Y = dataset[:,5]

# create model
model = Sequential()

# Add the first layer
# input_dim=   has to be the number of input variables. 
# It represent the number of inputs in the first layer,one per column 
model.add(Dense(12, input_dim=5, activation='relu'))

# Add the second layer
model.add(Dense(8, activation='relu'))

# Add the output layer
model.add(Dense(1, activation='sigmoid'))

# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Fit the model
model.fit(X, Y, epochs=150, batch_size=10)

# Evaluate the model
scores = model.evaluate(X, Y)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

# Save the model
model.save('model_file.h5')

Here's the code for predicting :

import numpy as np 
from keras.models import load_model # To save and load model

# Load the model
my_model = load_model('model_file.h5')

# Load the test data file and make predictions on it
predictions = my_model.predict(np.loadtxt("modiftest.csv", delimiter=";"))

print(predictions.shape)

my_predictions=my_model.predict(predictions)

print(my_predictions)

Here comes the error :

Traceback (most recent call last):
  File "predict01.py", line 14, in <module>
    my_predictions=my_model.predict(X_predictions)
  File "C:\Users\Philippe\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\models.py", line 913, in predict
    return self.model.predict(x, batch_size=batch_size, verbose=verbose)
  File "C:\Users\Philippe\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 1695, in predict
    check_batch_axis=False)
  File "C:\Users\Philippe\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 144, in _standardize_input_data
    str(array.shape))
ValueError: Error when checking : expected dense_1_input to have shape (None, 5) but got array with shape (200, 1)

Thank you for your help.

$\endgroup$
  • $\begingroup$ What's the shape of the array loaded by numpy? (In np.loadtxt('modiftest.csv')) $\endgroup$ – Mephy Dec 13 '17 at 11:23
  • $\begingroup$ When I execute this part of code : import numpy as np from keras.models import load_model my_model = load_model('model_file.h5') predictions = my_model.predict(np.loadtxt("modiftest.csv", delimiter=";")) print(predictions.shape) The result is : (200, 1) $\endgroup$ – Jed Dec 13 '17 at 13:01
2
$\begingroup$

I had a very similar issue when executing predict/predict_classes;

  ...
    classes = model.predict(XpredictInputData) 
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 1006, in predict
    return self.model.predict(x, batch_size=batch_size, verbose=verbose)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1772, in predict
    check_batch_axis=False)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 153, in _standardize_input_data
    str(array.shape))
ValueError: Error when checking : expected dense_1_input to have shape (None, 100) but got array with shape (100, 1)

The problem was occurring because the input array to keras predict was 1 dimensional (caused by a combination of a) numpy.genfromtxt producing a 1d array from a 1 row data file and b) a broadcasting failure). Keras predict appears to require a two dimensional array, even if there is only 1 prediction to be made.

I solved the issue by adding the following check before executing predict;

if (XpredictInputData.ndim == 1):
    XpredictInputData = numpy.array([XpredictInputData])
| improve this answer | |
$\endgroup$
1
$\begingroup$

The way you defined your model it expects an input of shape (None, 5) and will return an output of shape (m, 1) where m is the number of examples, i.e. the number of lines in the array you load.

So, for your data set you get an output of shape (200, 1) stored in your variable predictions. I don't know why you want to call the predict method on your predictions, but when you do it has to result in an error.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I understand that I have a problem of shape. This is the problem I'm trying to fix. I want to make predictions, with the model I have trained, on the data in modiftest.csv. That's what I'm trying to do when I call the predict method $\endgroup$ – Jed Dec 14 '17 at 14:57
  • $\begingroup$ I don't know what kind of data you are looking at exactly, but for training you split your .csv file into input X of shape (m, 5) and labels y of shape (m,1). $\endgroup$ – Lars Erik Würflinger Dec 14 '17 at 19:07
  • $\begingroup$ Your test data only seems to be of shape (p, 5), so there are no labels for your test set, right? Calling predict on the test input already gives you the predictions. So your variable predictions already contains the predictions for the test set. What else do you need? $\endgroup$ – Lars Erik Würflinger Dec 14 '17 at 19:16
1
$\begingroup$

@Jed let me explain your code step by step.

1.

#Load the model

my_model = load_model('model_file.h5')

This part loads the model created in previous section

2.

#Load the test data file and make predictions on it

predictions = my_model.predict(np.loadtxt("modiftest.csv", delimiter=";"))

This part uses test data and generates predictions on it. Your test data is size (,5) which means that there are 5 features in each test record.

After predicting, the predictions variable will have a single value for each test input.

So the shape of predictions array is (,1).

3.

print(predictions.shape)

This prints the predictions variable shape.

4.

my_predictions=my_model.predict(predictions)

This part is not needed because you already have predicted values from the model in predictions variable.

What you need to do:

So remove the part

my_predictions=my_model.predict(predictions)
| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks @ethan for giving new look to the answer.. $\endgroup$ – shantanu pathak Jun 22 '19 at 4:36

Not the answer you're looking for? Browse other questions tagged or ask your own question.