Selecting ML algorithm for music composition

Question

I'm a composer and programmer. I'd like to use ML for composing music. There's already research on the general problem of composing by machine in known musical styles. I'm more interested in the specific problem of using a computer to assist myself in composing... to speed up certain repetitive tasks, or to delight me with unexpected ideas.

First, I want to design ML that can make a good/bad categorization on a single short passage. (Not looking to generate music at first.)

It's pretty easier to decompose such a passage into significant features. I can say things like

• maybe there are particular "intervals" (harmony) present, and others absent
• maybe notes are spread out evenly from low to high, or maybe not
• maybe notes are clustered around a certain register rather than spread out
• maybe the rhythm is particularly uneven
• maybe a particular note occurs a lot

I can easily write algorithms that produce a list of such features for any given passage, and I can even be pretty sure that I'm including all of the relevant features. For an "art form," music is surprisingly amenable to analysis.

The trick is, first of all, there will be hundreds of observations, and second, that no one of these observations is good or bad in itself. Say we have three factors A, B, and C. A is not bad alone, but perhaps when A and B occur together, that's bad. But then maybe if A, B, and C all occur together, it becomes good again! And then maybe a fourth factor D, changes the context so that A, B, and C no longer matter much.

This seems like an ML problem, but I wonder what the best approach is?

I can construct or compose hundreds of examples of music labeled good and bad, and identify hundreds of "features" in each example. Some of the features are "present/not present" categories, while others are numbers that can be scaled from 0 to 1.

Is there a general and simple learning algorithm that even an ML novice such as myself (but I'm an experienced programmer) can implement?

I have to generate my own training examples and would be limited to around 300 to 1000, as I'm training it on simple passages in my style. This may actually be good problem to have... my job is not that complex compared to general musical intelligence.

I will contrast this task with image recognition. An image has millions of pixels and the features are spread out, occurring in endless permutations. It makes my head hurt to try to figure out what's going on in a deep neural net.

However, in music, there will be at most a few hundred features. The patterns will not occur in endless permutations. In fact, in stripped-down situations my wild guess is that you could make an 80% accurate classifier by looking at 10 features (assuming you have the weights and combinations well-trained). I'm aiming somewhat higher than that (I need more features and a bit more accurate).

To give you some idea of my data set: I'm in the early stages of writing the software to extract the features, but I have a pretty good mental idea of what I'm aiming for (been composing by hand for years and years). To make this more concrete I created a web page showing what the data will look like. This is very simple and just totally made up.

http://theunexpectedpearl.com/dataset.html

This shows four rows. Each row is an example passage of music that I will compose or generate automatically. Each row is classified "good" or "bad" by my own ear. The rest of the row is a long list of features, each of which is either "yes/no" or a single number scaled from 0.0 to 1.0.

I don't think a Naive Bayes classifier will work for this (if I understand correctly) because features are neither good nor bad in and of themselves... it is only combinations of them (context) that matter. In fact, the very same feature can contribute to "goodness" in one context, and be an immediate strong indication of "bad" in another context.

The data could possibly be structured differently. In practice, this spreadsheet would be "sparse" and maybe features should be grouped in some way, or arranged in a tree.

Answer 1

Let us formulate this problem in such a way that it can be understood from a machine learning perspective. You have a set of instances $X$ where each instance $x_i \in \mathbb{R}^m$ where $m$ is the dimensionality of the instance. In other words $m$ is the number of features that describe the instance. Your problem intends to go from a set of features to a class label good or bad. Thus, this is a mapping from $\mathbb{R}^m$ to $y \in \{0, 1\}$.

How to achieve this mapping?

This is when we will use the machine learning algorithms. We will train a model to effectively approximate the function which gives the output label from a set of inputs. It is evident that sparse features (low information entropy) will complicate the mapping function and will thus provide worse results. This is why feature engineering is of upmost importance for machine learning. It is probably the hardest part of the machine learning pipeline, however it is the lead factor in dictating your results.

You can use some feature reduction techniques in order to remove features which are uninformative with respect to the output label. Some techniques that I use frequently are principle component analysis (PCA), linear discriminant analysis (LDA). Alternatively, you can use some projection methods to reduce the dimensionality of the data whilst maintaining separation between the classes. Such techniques are Isomap, MDS, Spectral Embeddings and TSNE. You can check to see which is best suited for your type of data.

How to choose a model?

Firstly, your problem is a supervised classification problem. This already narrows the types of models you can use. Furthermore, model selection is based on some key factors such as: the number of instances you have, the number of features per instance and the number of output nodes. You should also keep in consideration that the separability of the probability distribution between the output classes will impact the performance of the model directly. For example discriminating between cars and oranges is much easier than oranges and clementines.

In your case, you have 1,000 instances and around 13 features. This means that deep learning based techniques are possible but discouraged. You do not have enough data. You can then attempt the following popular classification models

• Support Vector Classifier
• Naive Bayes
• K-Nearest Neighbors
• Decision Trees
• Random Forests

To evaluate which model performs the best you will use the accuracy attained with a trained model on a test set. This set should be drawn independently from the training set as to catch overfitting. This is when the model cannot generalize to new data.

---

In code

Assuming matrix $X$ contains the data where rows are the instances and columns are the features, and matrix $Y$ contains the labels.

First we split our data into a training and testing set

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.33)

from sklearn.svm import SVC
clf = SVC()
clf.fit(X_train, y_train)
print('Score: ', clf.score(X_test, y_test))

from sklearn.neighbors import KNeighborsClassifier
neigh = KNeighborsClassifier(n_neighbors=3)
neigh.fit(X_train, y_train)
print('Score: ', neigh.score(X_test, y_test))

from sklearn import tree
clf = tree.DecisionTreeClassifier()
clf.fit(X_train, y_train)
print('Score: ', clf.score(X_test, y_test))

from sklearn.ensemble import RandomForestClassifier 
forest = RandomForestClassifier(n_estimators = 100)
forest.fit(X_train, y_train)
print('Score: ', forest.score(X_test, y_test))

---

This should be a starting point. Let us know if you fall into any problems, and let us know what accuracy you are getting we can then look deeper into these models and better suit them to your data source.

Answer 2

I'll offer up a few suggestions to get the creative juices flowing.

First, since you have two tasks, you could consider two models configured as a Generative Adversarial Network where

• One model generates musical passages
• One model classifies musical passages

The output from the generative model is then evaluated by the classifier. The output from the classifier can then be used to improve (train) the generative model.

I imagine these would be trained on the same labeled data set. I would use a Recurrent Neural Network to train the classifier and tune the snot out of it (so to speak). I would use a Generative Neural Network for the generative model.

Maybe you were looking for more direction. If so, sorry to disappoint. I figured you may have been looking for better ways to search for help.

Best of luck.