I've noticed that in the Andrew Ng Deep Learning course that for image analysis he always has X_train matrices in the shape of [height, width, 3, num_inputs], or, if flattened, [height X width X 3, num_inputs]. He also has his y_train as [1, num_inputs]. To me, it is more intuitive to flip these so that X_train is [num_inputs, height X width X 3] and y_train is [num_inputs, 1]. Is there any motivating reason or justification that it has to be the way he does it or is it just preference? Is this a standard or does it vary?


2 Answers 2


It depends on the deep learning framework that you use, and you have to use the shape that the functions of the framework use. I think it is different in Tensorflow and Pytorch. The recommendation is to check before doing anything in the documentation of the framework.


In practice

Andrew Ng seems to be using the convention on the Theano framework. If you had 10 colour images, each 100 pixels high and 200 pixels wide, Theano models would expect input of the form:

(batch size, input channels, input rows, input columns)

and so dimensions: (10, 3, 100, 200). The three is because of the three RGb dimensions in a colour image.

Tensorflow on the other hand reverse this order to instead use: (num_obs, height, width, channels) - for the same example as above this becomes:

(batch size, input rows, input columns, input channels)

meaning the dimensions of the input should be (10, 100, 200, 3).

Keras works with Tensorflow and Theano and supports both conventions by simply allowing the user to set the position in which the number of channels are given. This can be set in a config file or using a specific environment variable upon setup. Have a look at the relevant documentation.

(Possible) justifications

There are different points of view on what is more natural. If you come from a computer vision arena (or image processing in general), then libraries like OpenCV use a coordinate system that has (0,0) in the top left of an image, and you specify a single pixel by given the vertical movement, then the horizontal movement from the origin. This means you give a height, then a width. One justification for this might be that many computer vision algorithms (e.g. colour filters) focus on difference across the channels of an image, but it is just convention.

In general graphing and mathematics, it is common to give the X coordinates first, then the Y, which means talking about horizontal movements, then vertical. Linear algebra in general is a good example of this.

All this being said, the best advice is to make sure you use the correct dimensions according to the documentation of the software you use. If you make your own software - you get to choose! (But make sure to document it!)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.