I am working on sentiment analysis using tweets text. I am able to build a word vector using Keras
text_to_sequence() method with pretrained GloVe embeddings in Python.
The question is how to add extra features beside the created GloVe embeddings and shape it in right way to feed to the embedding layer and the algorithm in general? For example, assuming the embedding's length is 100 and the extra features are 3, will appending the 3 features to 100 vector resulting in 103 vector length is the right approach? If so how would you implement it.
corpusAsList is the corpus as text in a list.
maxlen is the maximum number of columns to pad the word sequence with, so that all vectors have the same length.
maxVocabulary is the maximum amount of words to be used in the sequence and ignore the rest of the words.
tokenizer is used to represent a word as a digit in the vocabulary.
For instance, if we called
texts_to_sequences() like this:
# Create tokenized corpuss listTokenized = tokenizer.texts_to_sequences(corpusAsList) # Sequence dataset sequenceMatrixPrediction = sequence.pad_sequences(listTokenized, maxlen=maxlen)
And then trained a Bidirectional LSTM model like this:
def getLSTMModel(maxlen, maxVocabulary, embedSizeLSTM=100, weights=None): inp = Input(shape=(maxlen,)) x = Embedding(maxVocabulary, embedSizeLSTM, weights=[weights], trainable=False)(inp) x = Bidirectional(LSTM(80, return_sequences=True))(x) x = GlobalMaxPool1D()(x) x = Dropout(0.25)(x) x = Dense(70, activation="relu")(x) x = Dropout(0.25)(x) x = Dense(1, activation="sigmoid")(x) model = Model(inputs=inp, outputs=x) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) return model
How could we add three extra features to this model?