What could explain a much higher F1 score in comparision to accuracy score?

Question

I am building a binary classifier, which classifies numerical data, using Keras.

I have 6992 datapoints in my dataset. Test set is 30% of the data. And validation set is 30% of the training set.

When evaluating the model, I get these values:

recall:  0.8914240755310779
precision:  0.7006802721088435
f1_score:  0.7846260387811634
accuracy_score:  0.7035271816800843

How come is the accuracy_score so about 10% lower than the F1-score?

Here is the code I'm using to evaluate the model:

print('recall: ', recall_score(Y_test, y_pred))
print('precision: ', precision_score(Y_test, y_pred))
print('f1_score: ', f1_score(Y_test, y_pred))
print('accuracy_score: ', model.score(X_test, Y_test, verbose=0))

And here is my model:

def create_model(neurons=23):

    model = Sequential()
    model.add(Dense(neurons, input_dim=37, kernel_initializer='normal', activation='relu'))
    model.add(Dense(1, kernel_initializer='normal', activation='sigmoid'))

    # Compile model
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy', precision, recall])

    return model

model = KerasClassifier(build_fn=create_model, epochs=500, batch_size=5, verbose=1)

X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.3, random_state=seed)

transformer = Normalizer().fit(X_train)
X_train = transformer.transform(X_train)
transformer = Normalizer().fit(X_test)
X_test = transformer.transform(X_test)

tensorboard = TensorBoard(log_dir="logs/{}".format(time.time()))
time_callback = TimeHistory()
es = EarlyStopping(monitor='val_acc', min_delta=0, patience=20, verbose=0, mode='auto',restore_best_weights=True)

# Fit the model
history = model.fit(X_train, Y_train, validation_split=0.3, epochs=200, batch_size=5, verbose=1, callbacks=[tensorboard, time_callback]) 

Answer 1

You have imbalanced classes. Notice that your accuracy is very close to your precision, and quite dissimilar to your recall. This means that your precision (accuracy of positive predictions) is dominating the overall accuracy measure - nearly all of the cases in your data are classified as positive, so the accuracy among predicted positives is almost equivalent to the accuracy among all cases.

The F1 score is the harmonic mean of precision and recall, so it's a class-balanced accuracy measure. You have better performance on the minority class than the majority class, which is evidenced by the nearly equivalent accuracy and precision, and much higher recall.

Answer 2

F1-score is equal to:

2*((recall*precision)/(recall+precision))

If your model was catching more true negatives (TN) it would lower the F1-score. While your model is probably good at predicting true positives, it is likely predicting true negative at a lower rate.