Why does my chess neural network not generalize well?

Question

I'm trying to train a neural net to evaluate chess positions. For reference, the dataset I'm using can be found here. If you follow that link, you'll see that there are three CSV files - I'm only using the first two (chessData.csv and randomEvals.csv). My issue is that when I train the network on the data from one file, it doesn't perform well on the data from the other, even though both the training and testing set performance are adequate. The difference between the two files is that chessData.csv contains positions from official databases whereas randomEvals.csv contains positions reached by making random moves from positions in databases.

model.add(tf.keras.layers.Dense(2048, input_shape=(INPUT_DIM,), activation='relu'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dense(2048, activation='linear'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dense(2048, activation='relu'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dense(2048, activation='linear'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dense(2048, activation='relu'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dense(1, activation='linear'))

I'm representing the input as a 65 dimensional vector - each piece is represented by its value (1 for pawns, 3 for bishops/knights, 5 for rooks, 9 for queens, and 15 for kings) where values are positive for white pieces and negative for black ones. Also, I have one extra bit to represent the turn (1 for white, -1 for black). What are some suggestions to improve the generalization ability of this network?

Answer 1

As a start, you're using relu as your activation function on your input layer. This means any negative values will get clipped (i.e. your black pieces) to 0. Also, it's probably a good idea to scale your input values to 0..1. A quick glance at the kaggle comp, and you're probably going to want to encode things like castling possibilities as well as en-passant possibilities in your input as well. Seems like a fun problem to tackle.