Error when checking target: expected Output to have 2 dimensions, but got array with shape (631, 80, 2641)

Question

I'm making a Text Chunking program using Bi-LSTM from the model I of the paper "Neural Models for Sequence Chunking". The inputs are sequences of words and the outputs are "B-NP", "I-NP", and "O".

The problem I have is the output. The expected output is 2 dimensions, not 3 dimensions.

The X Train shape is (631, 80) while the Y Train shape is (631, 80, 2641).

I get the problem, but I don't know how to solve it. I've used np.array and reshape on Y Training, but no luck.

I don't have experience using Tensorflow and Keras and this is my first time, so I was hoping you could help me.

# Token Indexing
word2idx = {"PAD": 0, "UNK": 1}
word2idx.update({w: i+2 for i, w in enumerate(vocab_words) if w in vocab_words})
tag2idx = {t: i for i, t in enumerate(vocab_tags)}
idx2tag = {v: k for k, v in iteritems(tag2idx)}

# Padding
X = [[word2idx[w[0]] for w in s] for s in quran_sentences]
X = pad_sequences(maxlen=max_length, sequences=X, padding="post",value=word2idx["PAD"])

y = [[tag2idx[w[1]] for w in s] for s in quran_sentences]
y = pad_sequences(maxlen=max_length, sequences=y, padding="post", value=tag2idx["O"])
y = [to_categorical(i, num_classes=n_tags) for i in y]

# Split Data
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, shuffle=False)

# Parameters
hidden_state_encoder_size = 100

batch_size = 64

training_epoch = 200

embedding_size = 80

# Input
inputs = Input(shape=(max_length,), name="Input")

# Embedding
embed = Embedding(input_dim=n_words+2, output_dim=embedding_size, input_length=max_length, name="Embedding")(inputs)

# Bi-LSTM
encoder = Bidirectional(LSTM(units=hidden_state_encoder_size, return_state=True, name="LSTM"), name="Bi-LSTM")
encoder_outputs, forward_h, forward_c, backward_h, backward_c = encoder(embed)

# Concatenated
state_h = Concatenate(name="Concat_1")([forward_h, backward_h])
state_c = Concatenate(name="Concat_2")([forward_c, backward_c])

# Average
average = Average()([state_h, state_c])

# Outputs
outputs = Dense(num_decoder_tokens, activation="softmax", name="Output")(average)

# Build Model
model = Model(inputs, outputs, name="Seq2Seq Chunking")

# Compile & Training
# Compile
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')

# run training
model.fit(X_tr, np.array(y_tr),
          batch_size=batch_size,
          epochs=training_epoch,
          validation_split=0.1)

Error:

ValueError: Error when checking target: expected Output to have 2 dimensions, but got array with shape (631, 80, 2641)

Answer 1

Alright, so... I got help from someone. He mentions changing LSTM to return_sequences.

# Input
inputs = Input(shape=(max_length,), name="Input")

# Embedding
embed = Embedding(input_dim=n_words+1,
                  output_dim=embedding_size,
                  input_length=max_length,
                  name="Embedding")(inputs)

# Bi-LSTM
encoder = Bidirectional(LSTM(units=hidden_state_encoder_size,
                             return_sequences=True,
                             dropout=dropout_rate,
                             name="LSTM"),
                        name="Bi-LSTM")

encoder_outputs = encoder(embed)

# # Average
# average = Average()([state_h, state_c])

# Outputs
outputs = Dense(n_tags,
                activation="softmax",
                name="Output")(encoder_outputs)

The average layer is still wrong, but it's the close thing. I might have to edit it later.