2
$\begingroup$

I'm building a LSTM model to create an automatic drums composer. I'm following this post: LSTM Metallica

I've built my model and done all the enconding, I was able to emulate the behavior of the networks proposed in the post. In fact, if I use the dataset provided by the blog user (Metallica dataset) my model actually learns repeating patterns of musical bars.

Here's an example of the input encoding:

0b010000000 0b010000000 0b000000000 0b010000000 0b010000000 0b000001000 0b000000000 0b000001000 0b010000000 0b010000000 0b000000000 0b010000000 0b010000000 0b000001000 0b000000000 0b000001000 BAR

0b010000000 0b010000000 0b000000000 0b010000000 0b010000000 0b000001000 0b000000000 0b000001000 0b010000000 0b000000000 0b000000000 0b000001000 0b000000000 0b000001000 0b000001000 0b000000000 BAR

0b100000001 0b000000000 0b000000000 0b000000000 0b010000001 0b000000000 0b000000000 0b000000000 0b100000001 0b000000000 0b000000000 0b000000000 0b010000001 0b000000000 0b000000000 0b000000000 BAR 

0b100000001 0b000000000 0b000000000 0b000000000 0b010000001 0b000000000 0b000000000 0b000000000 0b100000001 0b000000000 0b000000000 0b000000000 0b010000001 0b000000000 0b000000000 0b000000000 BAR

0b100000001 0b000000000 0b000000000 0b000000000 0b010000001 0b000000000 0b000000000 0b000000000 0b100000001 0b000000000 0b000000000 0b000000000 0b010000001 0b000000000 0b000000000 0b000000000 BAR 

Where 0b100000001 represents the note played on the drums, while BAR represents the ending of a musical Bar. As you might notice, we have 16 events/notes per Bar.

If i train the network over the dataset provided, I get good results, my model learns the pattern/structure inserting a BAR every 16 notes predicted.

The loss of my model is the following: enter image description here

Which is the typical behavior of a network that starts overfitting at a certain point.

I tried the same model over my own dataset of songs, which I already know it's not that good (notes pattern do not make so much sense some time, we generated the drums artificially) but we still have a BAR every 16 notes.

I expected my network to poorly learn the drums generation but at least to learn the repeating pattern of BAR. So for example it generates totally random notes, but every 16 notes it generates a BAR

This unfortunately doesn't happen, the network has a totally random behavior, I tried to tune many parameters (input length, LSTM size. layers ecc...) without any success.

Also my network training has a strange behavior:

loss loss loss

These are the loss of the model over my dataset (using different parameters), and I can clearly notice that there's something wrong, the behavior of the model over the original dataset (Metallica, the first plot) had much more sense in my opinion.

Also thought of the underfitting/overfitting problem, but given the losses, both of them are really really low, and I can't understand why, given such low losses, my model hasn't learn anything at all from the data.

The model is the following:

model = Sequential()
model.add(LSTM(512, return_sequences=True, input_shape=(maxlen, num_chars)))
model.add(Dropout(0.2))
model.add(LSTM(512, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(512, return_sequences=False))
model.add(Dropout(0.2))
model.add(Dense(num_chars))
model.add(Activation('softmax'))
 
model.compile(loss='categorical_crossentropy', optimizer='adam')

Which could be the problem? I started to think that it's the dataset, but I expected my network to (at least) learn the repeating pattern of BAReven if the notes do not make much sense from the musical point of view.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.