0
$\begingroup$

I been fighting with this problem for 2 weeks now. And I extensively research for solutions here and in other sites. I have a dataset of 4 dimensions device_id, time and longitude and latitude. After preprocessing it look like this i got this shape

data1.shape
(95751, 4)

data1cnn2 = data1.values.reshape(1,95751, 4)

data1cnn2.shape

(1, 95751, 4)

y.shape

(95751,)

Then this is the code for the CNN1d . using Tensor-flow

modelcnn = Sequential()
modelcnn.add(Conv1D(filters=64, kernel_size=2, activation='relu', input_shape=(95751, 4)))
modelcnn.add(MaxPooling1D(pool_size=2))
modelcnn.add(Flatten())
modelcnn.add(Dense(50, activation='relu'))
modelcnn.add(Dense(1))
modelcnn.compile(optimizer='adam', loss='mse')

then I fit the model

modelcnn.fit(data2cnn, y, epochs=1000, verbose=0)

and I get this mistake


ValueError: Input arrays should have the same number of samples as target arrays. Found 1 input samples and 95751 target samples.

modelcnn.summary()

Layer (type)                 Output Shape              Param #   
=================================================================
conv1d_2 (Conv1D)            (None, 95750, 64)         576       
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 47875, 64)         0         
_________________________________________________________________
flatten_2 (Flatten)          (None, 3064000)           0         
_________________________________________________________________
dense_3 (Dense)              (None, 50)                153200050 
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 51        
=================================================================
Total params: 153,200,677
Trainable params: 153,200,677
Non-trainable params: 0
_

I also try with different solution but got different type of mistakes

I did

data1cnn = data1.values.reshape(95751, 4,1)

data1cnn.shape
(95751, 4, 1)

but i got this other mistake

ValueError: Error when checking input: expected conv1d_2_input to have shape (95751, 4) but got array with shape (4, 1)

modelcnn.summary()
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d_2 (Conv1D)            (None, 95750, 64)         576       
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 47875, 64)         0         
_________________________________________________________________
flatten_2 (Flatten)          (None, 3064000)           0         
_________________________________________________________________
dense_3 (Dense)              (None, 50)                153200050 
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 51        
=================================================================
Total params: 153,200,677
Trainable params: 153,200,677
Non-trainable params: 0

I am completely stuck and I have read all the question and answers that are here . With no success

$\endgroup$

1 Answer 1

1
$\begingroup$

Your first error is because Keras wants the batch size as first dimension. Given that your Y has 95751 observations, your X must be reshaped in order to have 95751 records in the first dimension, as you did in the second part.

The code doesn't work because Keras has already put the first dimension (None = batch_size) for you (see model.summary()), so you don't need to specify that in input_shape. You need to change this

input_shape=(95751, 4)

into this

input_shape=(4, 1)

Update after comment - working example

Here's a working example of your code.

First I define X and Y following your shapes. Here I use random numbers given that I don't have your data

data1cnn = np.random.rand(95751, 4, 1)
y = np.random.rand(95751,)

I check the shapes to be sure that they are the same as the ones you have provided in your second part of the question

print('X and Y shape')
print(data1cnn.shape, y.shape)

Output

X and Y shape
(95751, 4, 1) (95751,)

Looks correct. Now I define the model: everything is the same as in your code but input_shape=(4, 1)

modelcnn = Sequential()
modelcnn.add(Conv1D(filters=64, kernel_size=2, activation='relu', 
input_shape=(4, 1)))
modelcnn.add(MaxPooling1D(pool_size=2))
modelcnn.add(Flatten())
modelcnn.add(Dense(50, activation='relu'))
modelcnn.add(Dense(1))
modelcnn.compile(optimizer='adam', loss='mse')

I print the summary

print('Model summary')
print(modelcnn.summary())

Output

Model summary
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d_1 (Conv1D)           (None, 3, 64)             192       
_________________________________________________________________
max_pooling1d_1 (MaxPooling (None, 1, 64)             0         
_________________________________________________________________
flatten_1 (Flatten)         (None, 64)                0         
_________________________________________________________________
dense_1 (Dense)             (None, 50)                3250      
_________________________________________________________________
dense_2 (Dense)             (None, 1)                 51        
=================================================================
Total params: 3,493
Trainable params: 3,493
Non-trainable params: 0

Then I train it using the same command you wrote in your question but verbose=1 to print some information during training

modelcnn.fit(data1cnn, y, epochs=1000, verbose=1)

and it trains without a problem given that I get as output

Epoch 1/1000
95751/95751 [==============================] - 6s 60us/step - loss: 0.0844
Epoch 2/1000
95751/95751 [==============================] - 7s 76us/step - loss: 0.0833
Epoch 3/1000
95751/95751 [==============================] - 6s 59us/step - loss: 0.0832
Epoch 4/1000
95751/95751 [==============================] - 6s 66us/step - loss: 0.0831
Epoch 5/1000
95751/95751 [==============================] - 5s 54us/step - loss: 0.0831
Epoch 6/1000
95751/95751 [==============================] - 5s 54us/step - loss: 0.0830
$\endgroup$
4
  • $\begingroup$ Thank you for your answer I have done as you said and I still got error: ``` modelcnn.add(Conv1D(filters=32, kernel_size=2, activation='relu', input_shape=(4,1))) ValueError: Error when checking target: expected dense_4 to have shape (2,) but got array with shape (1,) ``` i have no enough space to add the model summary() $\endgroup$
    – Gabriel
    Aug 13, 2019 at 17:47
  • $\begingroup$ I can't help you without having a look at what you are doing. I put a working example in my answer following your description of the problem. Hope it helps $\endgroup$
    – black_cat
    Aug 13, 2019 at 20:53
  • $\begingroup$ Thank you. I have a very similar problem with an LSTM I will post tomorrow the question and send to you the link here $\endgroup$
    – Gabriel
    Aug 13, 2019 at 23:26
  • $\begingroup$ I work by the way $\endgroup$
    – Gabriel
    Aug 13, 2019 at 23:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.