I am trying to explore models for predicting whether a team will win or lose based on features about the team and their opponent. My training data is 15k samples with 760 numerical features. Each sample represents a game between two teams and the features are long and short term statistics about each team at the time of the game (i.e. avg points over last 10 games).
My thought was to use a binary classifier as a multi layered perceptron. Each layer has batch norm, dropout, and an ReLU activation. At the output layer there is a sigmoid activation. I am also using principal component analysis to reduce the dimensionality of my dataset.
I am using an automated hyper-parameter tuner with randomized search and population based training to find optimal accuracy of the hyper-parameters. I am able to tune the hyper-parameters such that training loss and validation loss both converge to a very small value, but when I look at accuracy I am getting very minimal improvements.
When looking at the initial accuracy after one epoch of training, it seems to be around 60% across all initial hyper-parameters, and it only reaches a max of about 70% after 30 epochs.
I am not really sure what to do. Does this imply my data is to noisy/random? Should I reconsider my architecture? Would I get better results with a regression instead of classification? Also, since the data is sequential in nature this might lend itself to a recurrent architecture, but I am not familiar with any RNN that is trained on multiple sequences as opposed to a single sequence.