0
$\begingroup$

I'm making a Text Chunking program using Bi-LSTM from the model I of the paper "Neural Models for Sequence Chunking". The inputs are sequences of words and the outputs are "B-NP", "I-NP", and "O".

The problem I have is the output. The expected output is 2 dimensions, not 3 dimensions.

The X Train shape is (631, 80) while the Y Train shape is (631, 80, 2641).

I get the problem, but I don't know how to solve it. I've used np.array and reshape on Y Training, but no luck.

I don't have experience using Tensorflow and Keras and this is my first time, so I was hoping you could help me.

# Token Indexing
word2idx = {"PAD": 0, "UNK": 1}
word2idx.update({w: i+2 for i, w in enumerate(vocab_words) if w in vocab_words})
tag2idx = {t: i for i, t in enumerate(vocab_tags)}
idx2tag = {v: k for k, v in iteritems(tag2idx)}

# Padding
X = [[word2idx[w[0]] for w in s] for s in quran_sentences]
X = pad_sequences(maxlen=max_length, sequences=X, padding="post",value=word2idx["PAD"])

y = [[tag2idx[w[1]] for w in s] for s in quran_sentences]
y = pad_sequences(maxlen=max_length, sequences=y, padding="post", value=tag2idx["O"])
y = [to_categorical(i, num_classes=n_tags) for i in y]

# Split Data
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, shuffle=False)

# Parameters
hidden_state_encoder_size = 100

batch_size = 64

training_epoch = 200

embedding_size = 80

# Input
inputs = Input(shape=(max_length,), name="Input")

# Embedding
embed = Embedding(input_dim=n_words+2, output_dim=embedding_size, input_length=max_length, name="Embedding")(inputs)

# Bi-LSTM
encoder = Bidirectional(LSTM(units=hidden_state_encoder_size, return_state=True, name="LSTM"), name="Bi-LSTM")
encoder_outputs, forward_h, forward_c, backward_h, backward_c = encoder(embed)

# Concatenated
state_h = Concatenate(name="Concat_1")([forward_h, backward_h])
state_c = Concatenate(name="Concat_2")([forward_c, backward_c])

# Average
average = Average()([state_h, state_c])

# Outputs
outputs = Dense(num_decoder_tokens, activation="softmax", name="Output")(average)

# Build Model
model = Model(inputs, outputs, name="Seq2Seq Chunking")

# Compile & Training
# Compile
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')

# run training
model.fit(X_tr, np.array(y_tr),
          batch_size=batch_size,
          epochs=training_epoch,
          validation_split=0.1)

Error:

ValueError: Error when checking target: expected Output to have 2 dimensions, but got array with shape (631, 80, 2641)
$\endgroup$
0
$\begingroup$

Alright, so... I got help from someone. He mentions changing LSTM to return_sequences.

# Input
inputs = Input(shape=(max_length,), name="Input")

# Embedding
embed = Embedding(input_dim=n_words+1,
                  output_dim=embedding_size,
                  input_length=max_length,
                  name="Embedding")(inputs)

# Bi-LSTM
encoder = Bidirectional(LSTM(units=hidden_state_encoder_size,
                             return_sequences=True,
                             dropout=dropout_rate,
                             name="LSTM"),
                        name="Bi-LSTM")

encoder_outputs = encoder(embed)

# # Average
# average = Average()([state_h, state_c])

# Outputs
outputs = Dense(n_tags,
                activation="softmax",
                name="Output")(encoder_outputs)

The average layer is still wrong, but it's the close thing. I might have to edit it later.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.