1
$\begingroup$

Im currently learning neural networks and I see conflicting decsriptions of the dimensions of weight and input matrices on the internet. I just wanted to know if there is some convention which more people use than the other.

I currently define my input matrix X with the dimensions of:

(m x n)

Where m is the number of samples and n is the number of features.

And I define my weight matrices with the dimensions:

(a x b)

Where a is the number of neurons in the layer and b is the number of neurons in the last layer.

Is that conventional or should I change something?

$\endgroup$

1 Answer 1

2
$\begingroup$

I would not say there is such a convention for it per se (if anyone has anything to comment on this, I would also like to know).

I think to make it clearer how the layer's input x interacts with the weights W, it might better to define the dimensions as the following:

  • x: (m x n)
  • W: (n x k)
  • bias term b: (k)

m remains as the number of examples. n represents number of input features and k represents number of neutrons in the layer.

As we know, we compute the output of the layer y as Wx + b. Therefore, the resulting output matrix will be (m x k)

$\endgroup$
2
  • $\begingroup$ Thanks! Guess I will switch to that notation for the weight matrices since I saw that one on the Internet too. I always thought it's a bit complicated since if we want to multiply W by X we have to write (W.T @ X.T).T to get a (m x k) output. $\endgroup$ Jul 6, 2020 at 9:43
  • $\begingroup$ Or just do X @ W, however that differs from most formulas I saw on the internet. $\endgroup$ Jul 6, 2020 at 9:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.