2
$\begingroup$

I have a question about the training example for machine learning. I know most people make training examples to a column vector. However, most of the time when we get a training example, like from a panda data frame object and transfer it to a NumPy array, each example will be a row vector. I know I can use transpose to change it to a column vector. But I want to ask why do we do that? What makes a column vector better suited for training examples?

$\endgroup$
1
  • $\begingroup$ A row vector is equivalent to a column vector; there's honestly no real difference. When working with dataframes (like pandas), it makes sense that each example is a row vector, because the values in each column should have the same datatype (for example string, float or integer), so each feature is a column and each example is a row. $\endgroup$ Aug 24 '20 at 10:53
2
$\begingroup$

We use column vectors because it offers efficient memory access in column-major format.

For example: CUDA, which is commonly used to accelerate libraries such as tensorflow, operates most efficiently when you coalesce memory. Strided memory access in parallelized applications cause significant slow-downs; threads (which are themselves sequentially indexed) should operate on sequentially indexed memory, rather than striding by a number of elements equal to the matrix row size. This is true even for cpu applications, mostly because of CPU caching. You want to make as few transactions as possible, which of course only column-major column-vectors or row-major row-vectors can offer.

Neither column-major nor row-major is universally "superior" and often they can be arbitrarily decided upon. However, there are a few reasons row-major is used in some applications but isn't used in others. In particular, relational databases are inherently row-major when you consider that each entry must be a single row. The C programming language is also inherently row-major because a 2D matrix is an array of arrays that stack row upon row when you consider the first dimension is the row index, as you do in math. The reasons for column-major format are to reflect mathematical interpretations, which also ties into some legacy integration; Fortran, for example, uses column-major where C is row-major. Fortran is still used in scientific computing applications, and although most have moved away from it, some of the most popular libraries originally written in Fortran have maintained a consistent API regardless of the language. BLAS, for example, was initially written in Fortran, but its many implementions in other language (including C/C++ for cuBLAS) have typically maintained its API. Many other popular mathematical programming tools, such as MATLAB, use column major format. It makes sense coming from a math-perspective where you consider the first dimension as the row index.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.