# Using text classification for system calls

I'm working on a project in which I should classify System calls sequences, my dataset is represented as sequences of integers (from 1 to 340). To do the classification I have inspired from Text classification projects. I'm trying to use on of them but I found a problem in my dataset shape, the code is:

df = pd.read_csv("data.txt")

#split arrays into train and test data (cross validation)
train_text, test_text, train_y, test_y = train_test_split(df,df,test_size =
0.2)

#train_text, train_y = (df,df)
#test_text, test_y = (df_test, df_test)
MAX_NB_WORDS = 5700

texts_train = train_text.astype(str)
texts_test = test_text.astype(str)

tokenizer = Tokenizer(nb_words=MAX_NB_WORDS, char_level=False)
tokenizer.fit_on_texts(texts_train)
sequences = tokenizer.texts_to_sequences(texts_train)
sequences_test = tokenizer.texts_to_sequences(texts_test)

word_index = tokenizer.word_index
#print('Found %s unique tokens.' % len(word_index))
type(tokenizer.word_index), len(tokenizer.word_index)
index_to_word = dict((i, w) for w, i in tokenizer.word_index.items())
" ".join([index_to_word[i] for i in sequences[0]])

seq_lens = [len(s) for s in sequences]
#print("average length: %0.1f" % np.mean(seq_lens))
#print("max length: %d" % max(seq_lens))

MAX_SEQUENCE_LENGTH = 100
# pad sequences with 0s
x_train = pad_sequences(sequences, maxlen=MAX_SEQUENCE_LENGTH)  # former des
sequence de meme taille 150, en ajoutant des 0
x_test = pad_sequences(sequences_test, maxlen=MAX_SEQUENCE_LENGTH)
#print('Shape of data train:', x_train.shape)  #it gives (1,100)
#print('Shape of data test tensor:', x_test.shape)
y_train = train_y
y_test = test_y
#if np.any(y_train):
#y_train = to_categorical(y_train)
print('Shape of label tensor:', y_train.shape)
EMBEDDING_DIM = 50
N_CLASSES = 2
# input: a sequence of MAX_SEQUENCE_LENGTH integers
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='float32')

embedding_layer = Embedding(MAX_NB_WORDS, EMBEDDING_DIM,
input_length=MAX_SEQUENCE_LENGTH,
trainable=True)
embedded_sequences = embedding_layer(sequence_input)

average = GlobalAveragePooling1D()(embedded_sequences)
predictions = Dense(N_CLASSES, activation='softmax')(average)

model = Model(sequence_input, predictions)
model.compile(loss='categorical_crossentropy',
model.fit(x_train, y_train, validation_split=0.1,
nb_epoch=10, batch_size=100)
output_test = model.predict(x_test)
print("test auc:", roc_auc_score(y_test,output_test[:,1]))


I got the error: ValueError Error when checking target expected dense_1 to have shape(2,), but got array with shape(1,)

Any suggestion, cause I don't know how to proceed . Thank you

The problem seems to arise in this line.

predictions = Dense(N_CLASSES, activation='softmax')(average)


Here, N_CLASSES has the value of 2. Meaning you have 2 classes. But the y_train vector has shape ( 1, ).

• You need to convert the y_train vector to a one hot vector. Here's any example.

If y_train = [ [ 0 ] , [ 1 ] ] then its one hot vector would be [ [ 1 , 0 ] , [ 0 , 1 ] ]. The one hot vector has shape ( 2 , ) which is needed in the above code snippet. So make this edit.

y_train = keras.utils.to_categorical( y_train , N_CLASSES )

• Thanks for your answer. I tried this an I had this error: ValueError: Input arrays should have the same number of samples as target arrays. Found 1 input samples and 0 target samples – Kahina Feb 20 '19 at 13:02
• Make sure that you have the number of labels equal to the number of samples. – Shubham Panchal Feb 20 '19 at 14:34
• I have two classes which means there are two labels, or something else ? Sorry but I'm a beginner – Kahina Feb 20 '19 at 18:00
• No, you will count only the number of labels and not the number of one hot numbers – Shubham Panchal Feb 21 '19 at 1:56