I have the last couple of month worked with an regression problem, of turning a framed audio file into a set of mfcc features, for a speech recognition application
I tried a lot different network structures, Cnn, different normalisation techniques, different optimizer, adding more layers and so on..
but finally i've got some decent result, but i don't understand why.. What i did was i added a linear layer as output, and somehow that minimised the error tremendeosly, and bit puzzled why a linear layer would have that much effect?...
I mean am still tried to fit the actual output to the desired output?.. Why would the activation function matter here?... I mean the weight are being adjusted based on the error, so why is the neural network better at adjusting for the error when the output is linear rather than non-linear (such as: tanh, Relu).. ?