Is it real to use word's n-grams for Deep Neural Network?
E.g., sentences list contains in X_train dataframe with "sentences" column. I use tokenizer from Keras in the next manner:
tokenizer = Tokenizer(lower=True, split=' ') tokenizer.fit_on_texts(X_train.sentences) X_train_tokenized = tokenizer.texts_to_sequences(X_train.sentences)
And later I use padding:
X_train_sequence = sequence.pad_sequences(X_train_tokenized)
Also I use simple LSTM network:
model = Sequential() model.add(Embedding(MAX_FEATURES, 128)) model.add(LSTM(32, dropout=0.2, recurrent_dropout=0.2, activation='tanh', return_sequences=True)) model.add(LSTM(64, dropout=0.2, recurrent_dropout=0.2, activation='tanh')) model.add(Dense(number_classes, activation='sigmoid')) model.compile(loss='categorical_crossentropy', optimizer = 'rmsprop', metrics=['accuracy'])
In this case, tokenizer execution. In Keras docs: https://keras.io/preprocessing/text/ I see character processing only, but it is nt apprepriate for my case.
My main question: Can I use n-grams for NLP tasks with deep learning (not necessary Sentiment Analysis, any abstract NLP task).
Indeed, in many tutorials or books I doesn't see any remainder n-grams for text processing, only embeddings.
For clarification: I'd like to consider not just words, but combination of words - I'd like to try it for my task.