1
$\begingroup$

I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My supervisor for my project wants me to output a ROC and AUC graph, but ROC and AUC graphs are only really for binary classification problems when reading online articles; this is what it tells me.

I am new to this area but thought I'd spend my weekend giving this a go. So far, nothing online has been helpful.

My model predicts on a dataset with 9,339 images; however, the dataset is unbalanced, but I have addressed this in the model. In total, there are 25 different classes/Families an image could belong to. I have read up that you can produce a ROC graph for every possible class.

path_to_data = "dataset"

train_test_generate_batches = ImageDataGenerator()
batch = train_test_generate_batches.flow_from_directory(directory=path_to_data, target_size=(64,64), batch_size=10000)
imgs, labels = next(batch)
X_train, X_test, y_train, y_test = train_test_split(imgs/255.,labels, train_size=0.7, test_size=0.3)

num_classes = 25

model = Sequential()

#input layer
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(64,64,3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.25))
model.add(Dropout(0.5))
model.add(Dense(50, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer = 'adam', metrics=['accuracy'])

y_train_new = np.argmax(y_train, axis=1)
class_weights = class_weight.compute_class_weight('balanced', np.unique(y_train_new), y_train_new)

history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10,  class_weight=class_weights)
scores = model.evaluate(X_test, y_test)

Could anyone explain how I could produce the curves from this neural network? Any help would be appreciated.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.