I am learning a DeepNN to choose between three decisions in a simulation. Therefore, I can run the simulation as often as I want and can generate as many samples as I want. Based on this tutorial (https://pythonprogramming.net/openai-cartpole-neural-network-example-machine-learning-tutorial/), I generated a learning dataset by running 10.000 simulations and saving the simulations with the 10% highest rewards.
Making use of this training data, I try to learn a DNN. However, during the learning phase, the loss of the DNN decreases while accuracy stays constant. Further, accuracy is about 0.33, which is basically random guessing when having 3 options to choose from. In consequence, when applying my DNN to yet unseen examples, it performs about as good as the best strategy used solely (so it seems to learn at least what strategy is best to use when applying only one predefined strategy. And that is exactly what it does).
The thing is: In my learning dataset, each strategy is used in about 1/3 of the cases, so in the learning dataset, all decisions are made about identical often. How is this possible?
I already experimented with different layouts of the DNN. Currently, it is defined as:
model = tf.keras.models.Sequential([ tf.keras.layers.Flatten(input_shape=(10,1)), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(256, activation='relu'), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(3, activation='softmax') ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
The output of the learning phase looks like:
Train on 10777 samples Epoch 1/5 10777/10777 [==============================] - 2s 150us/sample - loss: 6.9044 - accuracy: 0.3320 Epoch 2/5 10777/10777 [==============================] - 1s 84us/sample - loss: 1.5840 - accuracy: 0.3340 Epoch 3/5 10777/10777 [==============================] - 1s 94us/sample - loss: 1.1936 - accuracy: 0.3383 Epoch 4/5 10777/10777 [==============================] - 1s 96us/sample - loss: 1.1279 - accuracy: 0.3367 Epoch 5/5 10777/10777 [==============================] - 1s 82us/sample - loss: 1.1144 - accuracy: 0.3327