I am currently playing with tensorflow, but can't seem to get a hold whether it usefull for my problem?

I need to create a neural network, that is capable of mapping input to output.

The way things are progression now, i haven't a single example in which this has been done, all types of problem tensorflow seem to solve are classification task, and not this kind of mapping problem... Is tensorflow able to do so?, do I have to use a different neural network framework to do so?, And if it is available could somebody show some code (not the MNIST example)

My task:

I am currently trying to make an neural network that takes in samples of audio files, and generates MFCC features from the samples. The MFCC features can be calculated manually, which I have done, so I know what the I output i seek is. MFCC feature is a feature vector is different real values numbers. I cannot be classified as class A or Class B, or doing so would greatly reduce the accuracy of the output, as you are fitting the output to predetermined "bins"...

  • $\begingroup$ Classification is also a mapping, what kind of mapping are you looking for? An example would be in place... $\endgroup$ – Jan van der Vegt Oct 13 '16 at 14:00
  • $\begingroup$ I added a bit more to the description.., hopes it clarifies some things... $\endgroup$ – J.Down Oct 13 '16 at 14:05

From your description, it seems that you are facing a regression problem, because you want your output to be certain values. This is different from classification problems, that have as output the probability of the input belonging to certain class.

The key to using neural networks for regression is that the output layer should have no activation, that is, it should be a linear layer. As pointed out by @JanvanderVegt, a common loss function for regression problems is the mean square error (MSE) between the current output and the MFCC features you computed.

If you google "tensorflow regression example" you can find dozens of complete examples, like this or this.

  • $\begingroup$ And thanks for clarifying what I am actually searching for :P $\endgroup$ – J.Down Oct 13 '16 at 16:01
  • $\begingroup$ @ncasas but those examples are linear regression! I think an MLP example would be much better $\endgroup$ – Euler_Salter Oct 18 '17 at 9:24
  • $\begingroup$ Here you go: bfy.tw/EXNE $\endgroup$ – ncasas Oct 18 '17 at 10:23

What you propose is certainly possible, you would just have n nodes in your output layer, with n being the size of your MFCC feature vector, and you need to define a loss function that determines what mistakes it is currently making. If all the features have a similar scale and similar importance you could use the Mean Squared Error but you can get creative here. In principle this is not very different from multiclass classification with a softmax layer at the end. In that case you have c output nodes after a softmax layer and a cross-entropy loss function, in your case you have n nodes in your output layer after for example a fully connected layer with a MSE loss function. Feed it a number of examples and it should do the task reasonable if the data is represented well. Although I'm uncertain why you would want to use a ML approach if this is calculatable, you will lose accuracy and I'm uncertain how much speed you win.

A very easy example where TensorFlow is used for a regression task and not a classification task is linear regression with gradient descent, which you can see here: https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/2_BasicModels/linear_regression.ipynb

  • $\begingroup$ I am currently working on speech recognition, and trying to convert the general flow into something that uses dnn.. But do you have any example code on this.. I am currently unable to find any examples MNIST related. $\endgroup$ – J.Down Oct 13 '16 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.