1
$\begingroup$

👋

I'm currently writing my Master's Thesis on Subjective tagging of sounds and I feel that I've been stuck with the same problem for quite a time now and need assistance to progress. I'll, in short, describe my goal and what I've accomplished so far.

I'm writing a program that lets users manually tag audio files with a single label according to their own, i.e. subjective, perception. The tags are always adjectives, like happy, spooky or spacy. The goal is to have the program automatically tag similar sounds with the correct label by finding related sounds using machine learning. The program currently looks like this: Current UI of the program Usage of the program goes like this:

  1. The user adds sounds to be labeled.
  2. The program analyses the files and searches for related sounds using Algorithm 1.
  3. The user tags a sound and the program automatically tags related sounds using Algorithm 2.
  4. The user can verify tags given by the program and choose to accept them or give the sound a new tag.
  5. Repeat from step 1 or step 3.

By looking at the program flow above I've been able to implement step 1 using AudioCommons Timbral Models to convert sounds to a usable data format, and step 2 with Algorithm 1 using Mean shift to find groups of related sounds. The idea is to use the groups found by Algorithm 1 as a starting point when no tag has yet been given to a group. When one of the sounds in the group is tagged, all the sounds receive the same tag.

My problem, however, lies in step 3 with Algorithm 2 and step 4. I've tried a few different approaches, but they all seem a bit off and not quite fit for the job.

My goal is to have the program continuously or recursively learn from user validation, hence improving over-usage. My thought process is to have a verified flag on each sound, which the user can use to say if a tag given to a sound by the program is correct or not. Algorithm 2 should (re)learn from verified sounds in step 4 so that when step 3 is repeated, the automatic tagging of sounds done by the program is more accurate according to the user's choices.

So my question is: What kind of algorithm(s) is suitable for step 3 and 4 in place of Algorithm 2?


I've tried to make myself as clear as possible, but I'm quite new to machine learning, so feel free to point out confusing sections or errors. 🙂

In hope of assistance,

Andreas


Notes:

  • I've used tag and label interchangeably throughout the text; they refer to the same thing.
  • I want all machine learning to occur in isolation for a user, i.e. I don't want to learn from other user's input (like Collaborative filtering).
$\endgroup$
0
$\begingroup$

Just a few observations from my side:

  1. I think you should clarify your goal for yourself. Writing a program isn't a goal in itself. I'm missing the top level picture.
  2. The second part: 'The goal is to have the program automatically tag similar sounds with the correct label by finding related sounds using machine learning.' is more like a goal, but where is the subjective part in it, and how does this answer questions in your thesis?
  3. I'm not sure about the meaning of the output of Algorithm 1. Is this supervised or unsupervised classification? If it would be supervised, the 'supervision' is not subjective (until Algorithm 2). If is it unsupervised, the classes will find similarities, but the similarities are unlikely to correspond to the labels you intend to use.
  4. However, if you take e.g. $k=9$ classes with k-means (you will have to optimize the number) the similarities in the classes will increase, and you could 'learn' the matching of labels and the classes. The $k$ classes should map to one of 3 labels.
  5. Don't try to fit all these steps into one model. Create a few and connect them properly. Measure, interpret and report intermediate steps in your thesis.
$\endgroup$
10
  • $\begingroup$ Thank you for the answer! I'll try to respond to your observations 🙂 $\endgroup$ – Andreas Nasman Jan 20 '20 at 15:08
  • $\begingroup$ 1. I'm trying to create a solution that helps sound engineers categorise and organise their sounds by using subjective tags. My approach would be to create a program to minimise the manual effort needed to tag each sound individually. One way to measure the efficiency would be to compare the number of interactions needed from a user to achieve a state of completeness, i.e. when all sounds have the correct tag according to the user. In my thesis I would have examples where the state of completeness is defined and compare the clicks needed to achieve this state vs. tagging each manually. $\endgroup$ – Andreas Nasman Jan 20 '20 at 15:18
  • $\begingroup$ 2. By subjective I mean that I could tag say 10 sound examples as happy and sad, where you could say that they are spacy, chimy and dissonant. Each user could tag the same sounds with different labels and and with a varying number of labels, resulting in different amount of groups. This helps me answer the question of whether or not I can use e.g. Collaborative filtering in my program, which I can't in this case. $\endgroup$ – Andreas Nasman Jan 20 '20 at 15:24
  • $\begingroup$ 3. Shoot, forgot to mention it! All data is unlabelled to start with, so Algorithm 1 is dealing with unlabelled data completely. The reason I'm trying to implement Algorithm 1 is to find naturally occurring groups of sound, which would be used to create a "shortcut" for Algorithm 2 so it would be have something to start with, thus generating the correct tag faster. I don't know if this is a valid approach; it was just something that seemed logical to me and that I've been trying out. $\endgroup$ – Andreas Nasman Jan 20 '20 at 15:31
  • $\begingroup$ #2. Are your labels disjoint? E.g. Happy and Upbeat both allowed for the same sample? $\endgroup$ – Pieter21 Jan 20 '20 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.