0
$\begingroup$

I have a relatively simple 16 feature neural network attempting to predict the outcome of a sports event as win, loss, draw, however regardless of the number of layers, or the number of nodes in said layers, or the learning rate, etc; it still only predicts home and away wins, it never predicts a draw, and even then seems to disproportionately predict home wins. I have a good amount of data, c300,000 matches. I currently have 4 layers in my network, with v.low learning rate (0.000001), 0.001 L2 reg, 0.05 dropout between layers, and test between 60 and 2000 nodes per layer, but no configuration I can find will predict draws in the output. The labels are approx 45% home wins, 25% draws, and 30% away wins. I can show code as required.

Secondly, I'm also unsure if these accuracy and loss metrics show that I'm overfitting, or if the network is still learning. It appears to get to c48% accuracy relatively quickly then make tiny improvements, but I'm really not sure if those improvements are just overfitting.

May be worth noting that I have tried training various network iterations on single class data, and it appeared to train to each class pretty quickly. Otherwise the only things I've tried is changing the number of layers, nodes, hyperparameters, etc.

The first bar chart shows the classes output by the model on the basis of the test_x data - note it only ever predicts home win or away win, never a draw. The second bar chart shows the frequency of the test_y classes - I worry that fact that I'm nowhere near these suggests I have more fundamental issues? Any clarifications or uncertainty just let me know and I'll respond as quickly as possible.

This first bar chart shows the classes predicted by my model on the test set; the second shows the actual class balance in the test set.

Chart of predicted classes, and actual class frequency

Regarding the possible overfitting, here are my graphs of loss and accuracy:

Class balance

Any help is great appreciated!!

Improve this question
$\endgroup$
5
  • $\begingroup$ train and val loss are suspiciously close; I'd say train for another 100 epochs mb. How long does it take? $\endgroup$
    – Alex
    Commented Jan 12, 2021 at 11:19
  • $\begingroup$ It doesn't take very long, measurable in minutes. I'm currently running it to train over 1000 epochs, and the trend appears to continue - tiny increases in accuracy, tiny decreases in loss. In what sense do you mean 'suspiciously'? $\endgroup$
    – Cheeseburger
    Commented Jan 12, 2021 at 11:34
  • $\begingroup$ Typically val loss is greater than train $\endgroup$
    – Alex
    Commented Jan 12, 2021 at 12:20
  • 1
    $\begingroup$ Can you provide code that shows your network structure? $\endgroup$
    – Oxbowerce
    Commented Jan 12, 2021 at 15:41
  • $\begingroup$ Hi both, I managed to make some progress on this by balancing the classes. It APPEARS to be doing a better job now. Thanks again for your input! $\endgroup$
    – Cheeseburger
    Commented Jan 15, 2021 at 11:49

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.