I've built a binary classification model based on Keras and I am getting about 70% accuracy, and about 72% precision and 88% recall, making up to 79% F1-Score. I've tried different data models (deriving new features (columns) from the original ones), but the same model (same hyperparameters, same number of hidden layers (1), and same number of nodes in each layer).
For the different models created, after evaluating, the values of accuracy, precision, recall and F1-Score are almost the same as above. However, the Recall was always (for all models) high for all of the models tested, ranging from 85% to 100%.
What does that say about my model? Is it good enough? Why is the Recall for all models so high, but precision around 61% and 73%?
PS: When I mean "all models", as I said above, it is the same keras model, with same layers and nr of nodes and hyperparameters, just trying different dataframe as input. The first one is the original one, and the others have for example 1 feature less, one that I think is not relevant. Or another model, with input data same as original one, but with adding a derived feature, for example adding a feature "is_weekend", denoted with 1 or 0, but this feature is derived from the date feature already available in the original data. And for me, it's another model, as the input data is not the same.
Here are the evaluation results for my first model, the basic dataframe, with original features, no derived columns in it:
basic_df accuracy: 0.7068636796381271 f1-score: 0.7915721938539619 precison: 0.6974898160810807 recall: 0.9234265146423681
My dataset in not strongly imbalanced. I have 6992 data points, with 4303 class 1 (61.54%) and 2689 class 0 (38.46%).