First of all we have a classification task.
So we use the typical softmax cross entropy to classify.
Current implementation of curriculum learning is as follows.
- First we train our best version of the neural net
- At the last epoch we get all of the cross entropies for all the instances. So smaller cross entropies are better classified and larger ones not so well.
- Then we sort the instances based on the cross entropies.
- Then we start training the instances from easier to harder as the curriculum learning theory suggests
Note that we have already experimented with various steps and repetitions. So in one example we took the first 200 batches and trained them two times before going to the next batch and so on until an epoch is completed.
In another example we took the first 10 batches and trained them only once and then the next 10 and the next 10 and so on until the end of the epoch.
All of the experiments so far have concluded that the neural network is has a relatively ok accuracy at the beginning and this gets worsen as the more difficult instances come along. The final accuracy is much worse than expected and in addition the maximum accuracy is still quite bad.
Why is this curriculum learning not working? Is anything missing?