0
$\begingroup$

Lets say we have weights(theta1 and theta2) of neural net as:

theta1 =[1, 2, 3]
theta2= [4, 5, 6]

If we unroll these weights into a single dimension array in matlab/octave ,we get:

theta = [theta1(:);theta2(:)]
%theta = [1, 4, 2, 5, 3, 6]

If we unroll these weights in a slight different way , for example consider this python code:

theta = np.array(theta1,theta2)
  theta = theta.ravel()
  #theta = [1,2,3,4,5,6]

I have implemented a neural network using gradient descent in octave and it worked fine but the same implementation is not working in python(% accuracy = 10%). The only thing different the python implementation is the way the weights are unrolled. So, does the performance of neural network really depend on the way the weights are unrolled ?

$\endgroup$
  • $\begingroup$ Probably you should show a little more of the logic. You need to be consistently different with any code that uses these "unrolled" weights - presumably you are using them in some linear equations when training and calculating output. If you show that, probably someone can point out where your misunderstanding is. $\endgroup$ – Neil Slater Nov 3 '16 at 10:31
1
$\begingroup$

I don't really understand your notation and don't have a lot of experience with unrolling, but from my understanding there can only be one way of transforming your 2 weights (theta1 & theta2) to a single one (theta).

If you adapt your python code to something like:

theta = np.array(theta1,theta2)
theta = np.ravel(theta, order='F')
#theta = [1, 4, 2, 5, 3, 6]

Does it work now? If so, you can't just choose how to "unroll" your weights.

| improve this answer | |
$\endgroup$
  • $\begingroup$ It is not working for arrays with different dimensions. >>> a = np.array([[1,2,3,4],[4,5,6,7]]) >>> b = np.array([1,2,3]) >>> c = np.array((a,b)) >>> c = np.ravel(c,order='F') >>> c array([array([[1, 2, 3, 4], [4, 5, 6, 7]]), array([1, 2, 3])], dtype=object) $\endgroup$ – Saksham Nov 3 '16 at 15:39
  • $\begingroup$ But this isn't possible in matlab either right? Maybe you can add some code as an example? BTW, are speaking about a Recurrent Neural Network? For ordinary neural networks I don't see why you would combine all weights into a single array. $\endgroup$ – Laurens Meeus Nov 3 '16 at 16:32
  • $\begingroup$ well I am not very experienced in the field of machine learning.I am just following the coursera course on machine learning by andrew Ng . The example i that course uses octave and that implementation combined all weights into single array and used fmincg optimization function. Now I am trying to reimplement it in python . So yeah i dont know what is the difference between recurrent neural network and the normal neural network. Can you elaborate ? $\endgroup$ – Saksham Nov 3 '16 at 17:33
  • $\begingroup$ I think you made an error somewhere else. If you do: theta = [theta1(:);theta2(:)] %theta = [1, 2, 3, 4, 5, 6] you do get 1,2,3...6. Also you might want to check out this page: stackoverflow.com/questions/18801002/fminunc-alternate-in-numpy If you want to continue asking questions, please provide some code. $\endgroup$ – Laurens Meeus Nov 4 '16 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.