Just a quick question: I am building an ML model right now, however, I am receiving very similar (72.2 and 72.4 for example)% for both Accuracy and F1-Score on my Validation Dataset and my unseen Test Dataset, respectively. This is occurring on most of the baseline models I have produced for my problem right now.
Is this showing that my model is completely overfitting or just acting completely random and getting lucky?
Thanks
If the training set, validation set and the unseen test set (as you put it) have the same score, but lower than you expected then the model has not overfitted.
An overfitted model would have higher scores for the training data at least, and depending on how you optimised the hyper parameters on the validation data but lower for the unseen test set.
The more likely outcome is that your model has underfitted i.e. low but consistent scores across all 3 sets of data.
The existing answer is a good one - as it mentions you're likely underfitting. What type of model(s) are you using? If for example you're using Linear Regression you might opt for Polynomial instead. Or if the latter is being used you might incorporate some interaction terms . Finally if these do not suffice you might consider richer models like _splines_
