# When applying softmax over multiple dimensions of a tensor, does the order in which those dimensions are matter?

Lets say i have a tensor of order 256, dimensions indexed from 0 to 255.

Lets say i am writing a function implementing the softmax operation because i am a newbie and i want to understand the fundamental principles of machine learning.

Lets say the softmax is to be applied to dimensions 2 51 and 139.

Will the final result stay the same, regardless of whether i do softmax(2,51,139) or softmax(51,2,139) or softmax(139,51,2)?

Disregard the obvious problem of floating point operations not being really commutative.

• Hi @Maciej, welcome to the site. If you find the answers to your question useful, please consider upvoting them. Also, please consider accepting one (with the tick mark ✓ next to it) if you consider it correct or, alternatively, please describe in a comment why you consider it incorrect or not clear enough.
– noe
Apr 1 at 11:11

## 1 Answer

Softmax is applied along a single dimension, not along many dimensions.

For instance, in the Pytorch docs you can see that softmax receives param dim to specify the dimension over which it is:

Parameters

dim (int) – A dimension along which Softmax will be computed (so every slice along dim will sum to 1).

If you are referring to applying softmax sequentially over different dimensions, then the answer is that the order certainly matters. In the final result, only the dimension to which the softmax was applied last would be normalized.