LSTM neural network for music generation

Question

I am doing a project where I have to train a LSTM neural network to generate music. The first step is of course training it. I am using Keras on top of Theano for this task and music21 for feature extraction from MIDI files. So far the data I extract from MIDI files has the following format:

[[69, -7, 1, 0.75], [62, -1, 0, 0.25] ... ]

These are the 4 features that I take from the files. They are then placed in a 3D numpy array with the shape - (number of notes in sequence, 1 (time steps per sample), 4 (features) ). I create another array with the expected output for each time step being the data in the next time step. I've built the following model: http://pastebin.com/AHCJj1wa

The problem is that when it starts training the model I get some nonsensical output. First the epochs' output looks like this:

Epoch 1/100
0s - loss: 15.2986 - acc: 0.9738
Epoch 2/100
0s - loss: -3.3534e+02 - acc: 0.8265
Epoch 3/100
0s - loss: -3.8183e+02 - acc: 0.8265
Epoch 4/100
0s - loss: -3.8218e+02 - acc: 0.8265
Epoch 5/100
0s - loss: -3.8256e+02 - acc: 0.8265

It seems to be taking 0s per epoch and the loss just becomes larger and larger or stops changing at all (differs between runs). Also at the end of the training cycle when I call .predict it prints something like that:

[[ 0.11846249 -0.13457553  0.08614471 -0.1007532 ]
 [ 0.14596225 -0.17533173  0.10007301 -0.08799653]
 [ 0.15234099 -0.19707087  0.10395571 -0.08419057]
 [ 0.14596225 -0.17533173  0.10007301 -0.08799653]
 [ 0.14340727 -0.1664145   0.0987886  -0.09037785]
 [ 0.14596225 -0.17533173  0.10007301 -0.08799653]
 [ 0.14340727 -0.1664145   0.0987886  -0.09037785]
 [ 0.14802463 -0.17770432  0.10296948 -0.08740482]
 ..... ]

I think that there is something fundamentally wrong with my model and/or representation of the data. Any ideas on what I can do or at least some pointers to where I can read about this?

Answer 1

These are some suggestions you can try:

1. Just use 1 LSTM layer instead of 2, there might be less problem of vanishing gradient for shallower networks. In fact the dense layer of 4 is optional, since return_sequences = True.

2. Find a way to normalize the input in the range 0 to 1.

3. Try different optimizers like SGD or Adam.

4. Reduce your learning rate, the default learning rate might be too large for an input dimension of 4.

5. The loss should not be categorical_crossentropy, since this is a regression problem. The loss should mean squared error.

In order of importance 4 > 5 > 3 > 1 > 2 (i.e point 4 is most important)