# Running an LSTM with Music Data

I'm working on a project for a class where I'm trying to create an algorithm that learns music and creates its own music.

I'm having trouble on how to set up the data for it to be inputted into the LSTM.

A single training example consists of a chord that is a vector of binary values based on what keys are pressed in MIDI form (indices 0-127), a value that denotes duration of the note, beat strength, numerator of the time signature, and denominator of the time signature, and the key signature represented by the number of flats

So one example might look like

$$\left[ \begin{array}{c} {0} \\ {1} \\{0} \\{1} \\{\vdots} \\ {1} \\ {0} \\ {4} \\ {3} \\ {4} \\ {4} \\ {-2} \end{array} \right]$$

The result is a 132x1 vector

I was having trouble conceptualizing how to input this data type into an LSTM. Doing a linear output would not make that much sense, but I don't think I can directly one-hot this vector either.

## 2 Answers

You should ask yourself - are you teaching an algorithm to play chords or to play music? In addition, what are you trying to predict here?

It seems to me that you need to create input data that is a series of chords and your label is the next chord in the tune. So you should design a neural network that takes in a series of chords and can tell you the next chord in the sequence, add that back to the input sequence and pick the next chord, add that back to the input sequence, etc, etc. Next thing you know, you have a neural network that can play music.

If you are using Tensorflow, then create the input tensor of the dimension as given below :

input_data = tf.placeholder(tf.float32, [batch_size, timesteps, input_size], name='inputs')