0
$\begingroup$

I am learning a DeepNN to choose between three decisions in a simulation. Therefore, I can run the simulation as often as I want and can generate as many samples as I want. Based on this tutorial (https://pythonprogramming.net/openai-cartpole-neural-network-example-machine-learning-tutorial/), I generated a learning dataset by running 10.000 simulations and saving the simulations with the 10% highest rewards.

Making use of this training data, I try to learn a DNN. However, during the learning phase, the loss of the DNN decreases while accuracy stays constant. Further, accuracy is about 0.33, which is basically random guessing when having 3 options to choose from. In consequence, when applying my DNN to yet unseen examples, it performs about as good as the best strategy used solely (so it seems to learn at least what strategy is best to use when applying only one predefined strategy. And that is exactly what it does).

The thing is: In my learning dataset, each strategy is used in about 1/3 of the cases, so in the learning dataset, all decisions are made about identical often. How is this possible?

I already experimented with different layouts of the DNN. Currently, it is defined as:

model = tf.keras.models.Sequential([
            tf.keras.layers.Flatten(input_shape=(10,1)),
            tf.keras.layers.Dense(128, activation='relu'),
            tf.keras.layers.Dropout(0.2),
            tf.keras.layers.Dense(256, activation='relu'),
            tf.keras.layers.Dropout(0.2),
            tf.keras.layers.Dense(128, activation='relu'),
            tf.keras.layers.Dense(3, activation='softmax')
    ])
model.compile(optimizer='adam',
            loss='sparse_categorical_crossentropy',
            metrics=['accuracy'])

The output of the learning phase looks like:

Train on 10777 samples
Epoch 1/5
10777/10777 [==============================] - 2s 150us/sample - loss: 6.9044 - accuracy: 0.3320
Epoch 2/5
10777/10777 [==============================] - 1s 84us/sample - loss: 1.5840 - accuracy: 0.3340
Epoch 3/5
10777/10777 [==============================] - 1s 94us/sample - loss: 1.1936 - accuracy: 0.3383
Epoch 4/5
10777/10777 [==============================] - 1s 96us/sample - loss: 1.1279 - accuracy: 0.3367
Epoch 5/5
10777/10777 [==============================] - 1s 82us/sample - loss: 1.1144 - accuracy: 0.3327
$\endgroup$
3
  • $\begingroup$ Maybe you can include some more Dense layers and lower the learning rate to 0.0001. $\endgroup$ – Shubham Panchal Nov 26 '19 at 14:56
  • $\begingroup$ You have a deep network, why only 5 epochs? $\endgroup$ – Itamar Mushkin Nov 26 '19 at 15:01
  • $\begingroup$ I got the 5 epochs from an Tensor Flow example. However, it was a much smaller example than my task, so I agree that this was a wrong decision. I set a higher number of epochs. Also, I lowered the learning rate and added additional layers. $\endgroup$ – Xafer Nov 26 '19 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.