1
$\begingroup$

I have created an Artificial Neural Network with 4 features. I am at the point where I want to test the model with a live sample of a malicious file path/exe using:

new_prediction = classifier.predict(sc.transform(np.array([[]])))

I know that if I use the same File path like "C:\Program Files (x86\Wireless AutoSwitch" I could use "0" and so on for each of the categorical features that have already gone through onehot and label encoding. How do you deal with a new categorical feature that is not in the array of the training set? Lets say the new feature that I want to test is:

   ParentPath            ParentExe     ChildPath           ChildExe
0  C:\Windows\Malicious  badscipt.exe  C:\Windows\System   cmd.exe  

This training dataset looks like the following:

    ParentPath                                  ParentExe
0   C:\Program Files (x86)\Wireless AutoSwitch  wrlssw.exe
1   C:\Program Files (x86)\Wireless AutoSwitch  WrlsAutoSW.exs
2   C:\Program Files (x86)\Wireless AutoSwitch  WrlsAutoSW.exs
3   C:\Windows\System32                         svchost.exe
4   C:\Program Files (x86)\Wireless AutoSwitch  WrlsAutoSW.exs

ChildPath                                   ChildExe
C:\Windows\System32                         conhost.exe
C:\Program Files (x86)\Wireless AutoSwitch  wrlssw.exe
C:\Program Files (x86)\Wireless AutoSwitch  wrlssw.exe
C:\Program Files\Common Files               OfficeC2RClient.exe
C:\Program Files (x86)\Wireless AutoSwitch  wrlssw.exe
C:\Program Files (x86)\Wireless AutoSwitch  wrlssw.exe

The code:

#Libraries
import pandas as pd
import numpy as np
import hashlib
import matplotlib.pyplot as plt
import timeit

#################### GOOD ###################
#Read in csv to df
DF = pd.read_csv('/home/gpubetterwork/Documents/Good-Merged-TAGS_8-23- 
2018_060000-95959_TAG_Parent_Child.csv')
#Select 2 columns
DF1 = DF[['filePath', 'destinationProcessName']]
#Rename columns
DF1.columns = ['ParentPathExe', 'ChildPathExe']
#Replace all NaN with Unknown
DF1['ParentPathExe'] = DF1['ParentPathExe'].replace(np.nan, 'UNKNOWN')
DF1['ChildPathExe'] = DF1['ChildPathExe'].replace(np.nan, 'UNKNOWN')
#Split ParentPathExe into path and exe columns
DParent = DF1['ParentPathExe'].str.rsplit("\\", n=1, expand=True)
#Rename columns
DParent.columns = ['ParentPath', 'ParentExe']
#Split ChildPathExe into path and exe columns
DChild = DF1['ChildPathExe'].str.rsplit("\\", n=1, expand=True)
#Rename columns
DChild.columns = ['ChildPath', 'ChildExe']
#Merge the two dataframes together
DF1 = pd.concat([DParent, DChild], axis = 1)
#Fill new column DependentVariable with 0's
DF1['Suspicous'] = 0

####################### BAD ######################
BF = pd.read_csv('/home/gpubetterwork/Documents/4688_events_PC- 
Tags_last_7_days_BAD2.csv')
#Select 2 columns
BF1 = BF[['filePath', 'destinationProcessName']]
#Rename columns
BF1.columns = ['ParentPathExe', 'ChildPathExe']
#Replace all NaN with Unknown
BF1['ParentPathExe'] = BF1['ParentPathExe'].replace(np.nan, 'UNKNOWN')
BF1['ChildPathExe'] = BF1['ChildPathExe'].replace(np.nan, 'UNKNOWN')
#Split ParentPathExe into path and exe columns
BParent = BF1['ParentPathExe'].str.rsplit("\\", n=1, expand=True)
#Rename columns
BParent.columns = ['ParentPath', 'ParentExe']
#Split ChildPathExe into path and exe columns
BChild = BF1['ChildPathExe'].str.rsplit("\\", n=1, expand=True)
#Rename columns
BChild.columns = ['ChildPath', 'ChildExe']
#Merge the two dataframes together
BF1 = pd.concat([BParent, BChild], axis = 1)
#Fill new column DependentVariable with 1's
BF1['Suspicous'] = 1

############# MERGE GOOD AND BAD DATAFRAMES ###########
#Merge the two dataframes
DBF1 = DF1.append(BF1)
#Reset index
DBF1 = DBF1.reset_index(drop=True)  
#Randomize rows
DBF2 = DBF1.sample(frac=1).reset_index(drop=True)

############### ARTIFICIAL NEURAL NETWORK ##############
#TIME THE NEURAL NETWORK
start_time = timeit.default_timer()

#STEP 1
#Import the dataset
X = DBF2.iloc[:, 0:4].values
#X = DBF2[['ParentProcess', 'ChildProcess']]
y = DBF2.iloc[:, 4].values#.ravel()

#Encoding categorical data
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
#Label Encode Parent Path
labelencoder_X_1 = LabelEncoder()
X[:, 0] = labelencoder_X_1.fit_transform(X[:, 0])
#Label Encode Parent Exe
labelencoder_X_2 = LabelEncoder()
X[:, 1] = labelencoder_X_2.fit_transform(X[:, 1])
#Label Encode Child Path
labelencoder_X_3 = LabelEncoder()
X[:, 2] = labelencoder_X_3.fit_transform(X[:, 2])
#Label Encode Child Exe
labelencoder_X_4 = LabelEncoder()
X[:, 3] = labelencoder_X_4.fit_transform(X[:, 3])

#Create dummy variables
onehotencoder = OneHotEncoder(categorical_features = [0,1,2,3])
X = onehotencoder.fit_transform(X)

index_to_drop = [0, 1627, 2292, 5922]
to_keep = list(set(xrange(X.shape[1]))-set(index_to_drop))
X = X[:,to_keep]

#Splitting the dataset
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

#Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler(with_mean=False)
X_train_sc = sc.fit(X_train)
X_train = X_train_sc.transform(X_train)
X_test = X_train_sc.transform(X_test)

#STEP 2
#Make the ANN
import keras
from keras.models import Sequential
from keras.layers import Dense

#Initialising the ANN
classifier = Sequential()
#Adding the input layer and the first hidden layer
classifier.add(Dense(units=3678, kernel_initializer='uniform', 
activation='relu', input_dim=7356))
#Adding a second hidden layer
classifier.add(Dense(units=3678, kernel_initializer='uniform', 
activation='relu'))
#Adding the output layer
classifier.add(Dense(units=1, kernel_initializer='uniform', 
activation='sigmoid'))
#Compiling the ANN
classifier.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

#Fitting the ANN to the training set
classifier.fit(X_train, y_train, batch_size=1000, epochs=10)                           

#STEP 3
#Predicting the Test set results
y_pred = classifier.predict(X_test)
y_pred = (y_pred > 0.5)



##### NEW PREDICTION #####
#Must be in an array
new_prediction = classifier.predict(sc.transform(np.array([[]])))
new_prediction = (new_prediction > 0.5)
$\endgroup$
  • $\begingroup$ If we are going to down vote, can we at least explain why this is a bad question. Examples: This is common knowledge. I was working on the down vote badge. This question is unclear. I have not included the right tags. $\endgroup$ – sectechguy Sep 20 '18 at 15:39
  • $\begingroup$ How can you one hot if the classes are different? Maybe they might balance off each other somehow? $\endgroup$ – Aditya Sep 20 '18 at 15:58
  • $\begingroup$ Downvoting this question is not the best but unfortunately we have a huge tendency to downvote in DSEC! I edited your question. Your main problem is about having an unseen value in categorical feature while testing on unseen data. A pretty good question actually! $\endgroup$ – Kasra Manshaei Nov 2 '18 at 10:23
1
$\begingroup$

Just as a suggestion, if you insist on using Neural Networks and in your live demo you may encounter new values, then my suggestion would be having an "unseen" value (e.g. with all 0 entries in one-hot-encoding schema) in the categorical features. It gives a degree of freedom to the system not to raise any error or something when sees a new thing. As you need to vectorize your input through a preprocessing step, the unseen value can be mapped to this dummy value.

On the other hand you keep these unseen values and their manually annotated labels aside and, for a live system, retrain your model at certain time points (e.g. every night) using the new batch you already got so you can include them in your feature values this way. Of course in this case the performance of system improves by time as it will see more variations of data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.