I have a data set of total 8000 sound samples. These are the results of my multi layer neural network, binary classifier:
Precision: [0.95 0.96]
Recall: [0.96 0.95]
F-Score: [0.95 0.95]
S: [1217. 1254.]
Accuracy training 1.0
Accuracy Test 0.95
I am happy with the test accuracy, precision and recall. But the 1.0 accuracy bugs me since its overfitting to the training-set. Is this a bad thing even the test-set accuracy is satisfactory?
Here below I load the car sound features(1) and other types of sounds features(0) from disk and assign the labels. I double checked and I dont add labels as a feature while I process and extract features from audio. sets are cut to be equal to 4100 samples each
car_features =np.load('car_features_final.npy')
car_labels =np.ones(len(car_features),dtype=int)
shuffle(car_features, random_state=12)
other_features = np.load('other_features.npy')
other_features=list(islice(shuffle(other_features, random_state=12),4100))
other_labels = np.zeros(len(other_features),dtype=int)
shuffle(other_features, random_state=12)
all_features = np.append(car_features, other_features, axis=0)
all_labels = np.append(car_labels,other_labels, axis=0)
X_train, X_test, y_train, y_test = train_test_split(all_features, all_labels, test_size=0.30, random_state=42)
clf = MLPClassifier(activation='relu', solver='adam', alpha= 0.1, hidden_layer_sizes=(300, 300, 300, 100), random_state=1, max_iter=500)
clf.fit(X_train, y_train)
y_predicted_test = clf.predict(X_test)
y_predicted_train = clf.predict(X_train)
p,r,f,s = precision_recall_fscore_support(y_test, y_predicted_test)