Keras model looks like this

inp = Input(shape=(maxlen, ))
x = Embedding(max_features, embed_size, weights=[embedding_matrix], trainable=False)(inp)
x = SpatialDropout1D(dropout)(x)
x = Bidirectional(LSTM(num_filters, return_sequences=True))(x)
max_pool = GlobalMaxPooling1D()(x)
x = concatenate([x_h, max_pool])
outp = Dense(6, activation="sigmoid")(x)

According to the documentation the output shape = input shape i.e. (samples, timesteps, channels)


  1. What does SpatialDropout1D() really do to the output of Embedding()? I know the output of LSTM Embedding is of dimension (batch_size, steps, features).
  2. Does SpatialDropout1D() just randomly replace some values of word embedding of each word by 0?
  3. How is SpatialDropout1D() different from Dropout() in Keras?
  • $\begingroup$ Intuitively This version performs the same function as Dropout, however it drops entire 1D feature maps instead of individual elements.. $\endgroup$
    – Aditya
    Sep 20, 2018 at 4:00
  • 1
    $\begingroup$ What is a 1D feature maps in the context of Embedding()? $\endgroup$ Sep 20, 2018 at 4:02
  • $\begingroup$ stackoverflow.com/questions/50393666/… $\endgroup$
    – Aditya
    Sep 20, 2018 at 4:37
  • $\begingroup$ Dropping the cols itself! Is what I can conclude.. need to play around them like visualisation of the embedding to confirm it! $\endgroup$
    – Aditya
    Sep 20, 2018 at 8:49

1 Answer 1


Basically, it removes all the pixel in a row from all channels. eg: take [[1,1,1], [2,4,5]], there are 3 points with values in 2 channels, by doing SpatialDropout1D it zeros an entire row ie all attributes of a point is set to 0; like [[1,1,0], [2,4,0]]

number of such choices would be 3C0 + 3C1+ 3C2 + 3C3 = 8

The intuition behind this is in many cases for an image the adjacent pixels are correlated, so hiding one of them is not helping much, rather hiding entire row, that's gotta make a difference. reference material


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.