I have a text file with information that needs to classified based on keywords. The text file contains many number of paragraphs. And the paragraph contains keywords that we want (lets say salary amount, interest rate and so on..)

I want to write a model which will extract the paragraph (or 3 to 4 lines of text) containing the keyword i want. How do i create a label in this case? All i have is a raw text.

I am new to NLP. Any suggestions how i can approach this?


1 Answer 1


You can build the text classification application with CNN algorithm by Keras library. Please take a look at this git repository. Here

As you can see, you need to create training and testing data by loading polarity data from files, splitting the data into words, generating labels and returning split sentences and labels. And you can create the convolutional neural network with Dense, Embedding, Conv2D, MaxPool2D of keras.

Here is the final model training snippet.

from keras.layers import Input, Dense, Embedding, Conv2D, MaxPool2D
from keras.layers import Reshape, Flatten, Dropout, Concatenate
from keras.callbacks import ModelCheckpoint
from keras.optimizers import Adam
from keras.models import Model
from sklearn.model_selection import train_test_split
from data_helpers import load_data

print('Loading data')
x, y, vocabulary, vocabulary_inv = load_data()

# x.shape -> (10662, 56)
# y.shape -> (10662, 2)
# len(vocabulary) -> 18765
# len(vocabulary_inv) -> 18765

X_train, X_test, y_train, y_test = train_test_split( x, y,     test_size=0.2, random_state=42)

# X_train.shape -> (8529, 56)
# y_train.shape -> (8529, 2)
# X_test.shape -> (2133, 56)
# y_test.shape -> (2133, 2)

sequence_length = x.shape[1] # 56
vocabulary_size = len(vocabulary_inv) # 18765
embedding_dim = 256
filter_sizes = [3,4,5]
num_filters = 512
drop = 0.5

epochs = 100
batch_size = 30

# this returns a tensor
print("Creating Model...")
inputs = Input(shape=(sequence_length,), dtype='int32')
embedding = Embedding(input_dim=vocabulary_size,     output_dim=embedding_dim, input_length=sequence_length)(inputs)
reshape = Reshape((sequence_length,embedding_dim,1))(embedding)

conv_0 = Conv2D(num_filters, kernel_size=(filter_sizes[0],     embedding_dim), padding='valid', kernel_initializer='normal',     activation='relu')(reshape)
conv_1 = Conv2D(num_filters, kernel_size=(filter_sizes[1],     embedding_dim), padding='valid', kernel_initializer='normal',     activation='relu')(reshape)
conv_2 = Conv2D(num_filters, kernel_size=(filter_sizes[2],     embedding_dim), padding='valid', kernel_initializer='normal', activation='relu')(reshape)

maxpool_0 = MaxPool2D(pool_size=(sequence_length - filter_sizes[0] + 1, 1), strides=(1,1), padding='valid')(conv_0)
maxpool_1 = MaxPool2D(pool_size=(sequence_length - filter_sizes[1] + 1, 1), strides=(1,1), padding='valid')(conv_1)
maxpool_2 = MaxPool2D(pool_size=(sequence_length - filter_sizes[2] + 1, 1), strides=(1,1), padding='valid')(conv_2)

concatenated_tensor = Concatenate(axis=1)([maxpool_0, maxpool_1, maxpool_2])
flatten = Flatten()(concatenated_tensor)
dropout = Dropout(drop)(flatten)
output = Dense(units=2, activation='softmax')(dropout)

# this creates a model that includes
model = Model(inputs=inputs, outputs=output)

checkpoint = ModelCheckpoint('weights.{epoch:03d}-{val_acc:.4f}.hdf5',     monitor='val_acc', verbose=1, save_best_only=True, mode='auto')
adam = Adam(lr=1e-4, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)

model.compile(optimizer=adam, loss='binary_crossentropy', metrics=['accuracy'])
print("Traning Model...")
model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, callbacks=[checkpoint], validation_data=(X_test, y_test))  #     starts training

By running this code, you will get the trained model with the format of hd5. Finally, you can use your model for prediction.

  • $\begingroup$ I have a small doubt in labels. I want to take only particular keyword as labels and extract a paragraph. In that code, generating lables is done by "positive_labels = [[0, 1] for _ in positive_examples]" . In my case, i want to labels to be a particular word and not all words. How can i do this? $\endgroup$
    – dhinar1991
    Commented May 4, 2018 at 9:19
  • $\begingroup$ ah~ sorry. in order to get the prediction based on keyword, this example would be helpful rather than above source. github.com/inspirehep/magpie/tree/master/magpie $\endgroup$
    – Top Coder
    Commented May 4, 2018 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.