# Does the performance of neural networks depend on the method used to unroll weights ?

Lets say we have weights(theta1 and theta2) of neural net as:

theta1 =[1, 2, 3]
theta2= [4, 5, 6]


If we unroll these weights into a single dimension array in matlab/octave ,we get:

theta = [theta1(:);theta2(:)]
%theta = [1, 4, 2, 5, 3, 6]


If we unroll these weights in a slight different way , for example consider this python code:

theta = np.array(theta1,theta2)
theta = theta.ravel()
#theta = [1,2,3,4,5,6]


I have implemented a neural network using gradient descent in octave and it worked fine but the same implementation is not working in python(% accuracy = 10%). The only thing different the python implementation is the way the weights are unrolled. So, does the performance of neural network really depend on the way the weights are unrolled ?

• Probably you should show a little more of the logic. You need to be consistently different with any code that uses these "unrolled" weights - presumably you are using them in some linear equations when training and calculating output. If you show that, probably someone can point out where your misunderstanding is. – Neil Slater Nov 3 '16 at 10:31

## 1 Answer

I don't really understand your notation and don't have a lot of experience with unrolling, but from my understanding there can only be one way of transforming your 2 weights (theta1 & theta2) to a single one (theta).

If you adapt your python code to something like:

theta = np.array(theta1,theta2)
theta = np.ravel(theta, order='F')
#theta = [1, 4, 2, 5, 3, 6]


Does it work now? If so, you can't just choose how to "unroll" your weights.

• It is not working for arrays with different dimensions. >>> a = np.array([[1,2,3,4],[4,5,6,7]]) >>> b = np.array([1,2,3]) >>> c = np.array((a,b)) >>> c = np.ravel(c,order='F') >>> c array([array([[1, 2, 3, 4], [4, 5, 6, 7]]), array([1, 2, 3])], dtype=object) – Saksham Nov 3 '16 at 15:39
• But this isn't possible in matlab either right? Maybe you can add some code as an example? BTW, are speaking about a Recurrent Neural Network? For ordinary neural networks I don't see why you would combine all weights into a single array. – Laurens Meeus Nov 3 '16 at 16:32
• well I am not very experienced in the field of machine learning.I am just following the coursera course on machine learning by andrew Ng . The example i that course uses octave and that implementation combined all weights into single array and used fmincg optimization function. Now I am trying to reimplement it in python . So yeah i dont know what is the difference between recurrent neural network and the normal neural network. Can you elaborate ? – Saksham Nov 3 '16 at 17:33
• I think you made an error somewhere else. If you do: theta = [theta1(:);theta2(:)] %theta = [1, 2, 3, 4, 5, 6] you do get 1,2,3...6. Also you might want to check out this page: stackoverflow.com/questions/18801002/fminunc-alternate-in-numpy If you want to continue asking questions, please provide some code. – Laurens Meeus Nov 4 '16 at 9:35