# When doing forward propogation in DNN's for a $m\times n$ matrix, does $m$ or $n$ dictate the dimensionality of the output matrix?

I am tracing through matrix multiplication in forward propagation. For some context, I am thinking about a input shape of (1, 2, 100). I am curious how (2, 100) is propagated forward in a neural network.

I noticed that for a $$m\times n$$ matrix, $$m$$ is preserved throughout. Reading column wise, this would be a collection of vectors in $$m$$ dimensions. However, I am curious what $$n$$ contributes to the output matrix (or the matrices in between hidden layers)?

• Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
– Community Bot
Commented Jun 20, 2023 at 13:30

If so, $$m$$ represents your batch size so how many inputs you forward to your network: each vector has $$n$$ components. So you get an $$m\times n$$ input matrix.
The size $$m$$ must be preserved all along the layers of the network because is the batch size, so the number of inputs that you provided as inputs. Therefore, (dense) layers actually work on the $$n$$ dimension, shrinking or expanding it.
Say your network has one hidden dense layer, its weight matrix would be $$n\times u$$, because it works on the dimensionality (num of components) of the vectors, transforming them from $$n$$ to $$u$$, resulting in an output (or hidden) matrix of shape $$m\times u$$.
• @ajsdkh Right. Say you put a dense layer on top with $u$ units, so according to the "note" section here, the $(100, u)$ matrix is used as a kernel yielding $1\star 2$ sub-tensors, which are then reshaped to $(1,2,u)$. See also here for more details. Commented Jun 17, 2023 at 17:28