I have a data set of total 8000 sound samples. These are the results of my multi layer neural network, binary classifier:
Precision: [0.95 0.96] Recall: [0.96 0.95] F-Score: [0.95 0.95] S: [1217. 1254.] Accuracy training 1.0 Accuracy Test 0.95
I am happy with the test accuracy, precision and recall. But the 1.0 accuracy bugs me since its overfitting to the training-set. Is this a bad thing even the test-set accuracy is satisfactory?
Here below I load the car sound features(1) and other types of sounds features(0) from disk and assign the labels. I double checked and I dont add labels as a feature while I process and extract features from audio. sets are cut to be equal to 4100 samples each
car_features =np.load('car_features_final.npy') car_labels =np.ones(len(car_features),dtype=int) shuffle(car_features, random_state=12) other_features = np.load('other_features.npy') other_features=list(islice(shuffle(other_features, random_state=12),4100)) other_labels = np.zeros(len(other_features),dtype=int) shuffle(other_features, random_state=12) all_features = np.append(car_features, other_features, axis=0) all_labels = np.append(car_labels,other_labels, axis=0) X_train, X_test, y_train, y_test = train_test_split(all_features, all_labels, test_size=0.30, random_state=42) clf = MLPClassifier(activation='relu', solver='adam', alpha= 0.1, hidden_layer_sizes=(300, 300, 300, 100), random_state=1, max_iter=500) clf.fit(X_train, y_train) y_predicted_test = clf.predict(X_test) y_predicted_train = clf.predict(X_train) p,r,f,s = precision_recall_fscore_support(y_test, y_predicted_test)