I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My supervisor for my project wants me to output a ROC and AUC graph, but ROC and AUC graphs are only really for binary classification problems when reading online articles; this is what it tells me.

I am new to this area but thought I'd spend my weekend giving this a go. So far, nothing online has been helpful.

My model predicts on a dataset with 9,339 images; however, the dataset is unbalanced, but I have addressed this in the model. In total, there are 25 different classes/Families an image could belong to. I have read up that you can produce a ROC graph for every possible class.

path_to_data = "dataset"

train_test_generate_batches = ImageDataGenerator()
batch = train_test_generate_batches.flow_from_directory(directory=path_to_data, target_size=(64,64), batch_size=10000)
imgs, labels = next(batch)
X_train, X_test, y_train, y_test = train_test_split(imgs/255.,labels, train_size=0.7, test_size=0.3)

num_classes = 25

model = Sequential()

#input layer
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(64,64,3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dense(128, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer = 'adam', metrics=['accuracy'])

y_train_new = np.argmax(y_train, axis=1)
class_weights = class_weight.compute_class_weight('balanced', np.unique(y_train_new), y_train_new)

history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10,  class_weight=class_weights)
scores = model.evaluate(X_test, y_test)

Could anyone explain how I could produce the curves from this neural network? Any help would be appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.