I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My supervisor for my project wants me to output a ROC and AUC graph, but ROC and AUC graphs are only really for binary classification problems when reading online articles; this is what it tells me.
I am new to this area but thought I'd spend my weekend giving this a go. So far, nothing online has been helpful.
My model predicts on a dataset with 9,339 images; however, the dataset is unbalanced, but I have addressed this in the model. In total, there are 25 different classes/Families an image could belong to. I have read up that you can produce a ROC graph for every possible class.
path_to_data = "dataset" train_test_generate_batches = ImageDataGenerator() batch = train_test_generate_batches.flow_from_directory(directory=path_to_data, target_size=(64,64), batch_size=10000) imgs, labels = next(batch) X_train, X_test, y_train, y_test = train_test_split(imgs/255.,labels, train_size=0.7, test_size=0.3) num_classes = 25 model = Sequential() #input layer model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(64,64,3))) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Conv2D(16, (3, 3), activation='relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(128, activation='relu')) model.add(Dropout(0.25)) model.add(Dropout(0.5)) model.add(Dense(50, activation='relu')) model.add(Dense(num_classes, activation='softmax')) model.compile(loss='categorical_crossentropy', optimizer = 'adam', metrics=['accuracy']) y_train_new = np.argmax(y_train, axis=1) class_weights = class_weight.compute_class_weight('balanced', np.unique(y_train_new), y_train_new) history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, class_weight=class_weights) scores = model.evaluate(X_test, y_test)
Could anyone explain how I could produce the curves from this neural network? Any help would be appreciated.