Question Related Numpy

With Numpy, what’s the best way to compute the inner product of a vector of size 10 with each row in a matrix of size (5, 10)?

• np.array([np.dot(vec, row) for row in mat]) should be fine for most purposes. It helps if you define "best". Also it's generally a good idea to post pure programming questions on Stackoverflow, not here. – shadowtalker Oct 8 '18 at 15:20

Here are a few ways, using some dummy data:

In : import numpy as np

In : a = np.random.randint(0, 10, (10,))

In : b = np.random.randint(0, 10, (5, 10))

In : a
Out: array([4, 1, 0, 6, 3, 3, 6, 6, 1, 8])

In : b
Out:
array([[9, 0, 6, 1, 1, 1, 4, 7, 4, 7],
[5, 8, 8, 3, 4, 8, 7, 3, 0, 4],
[2, 2, 5, 3, 9, 6, 1, 5, 8, 3],
[2, 0, 4, 3, 5, 3, 3, 4, 3, 3],
[3, 3, 6, 4, 7, 5, 8, 6, 7, 3]])

Because of the dimensions you asked for, in order to compute inner products (a.k.a. scalar products and dot products), we need to transpose the matrix b so that the dimensions work out. With a vector of length 10, numpy gives it shape (10,). So it seems 10 rows and no columns, however it is kind of ambiguous. Numpy will essentially do what it has to in order to make dimensions work. We could force it into a (10, 1) vector by using a.reshape((10, 1)), but it isn't necessary. The matrix has a defined second dimensions, so we have a shape (5, 10). In order to multiply these two shapes together, we need to make the same dimensions match in the middle. This means making (10,) * (10, 5). Performing the transpose on matrix reverses the dimensions to give us that (10, 5). Those inner 10s will then disappear and leave us with a (1, 5) vector.

That all being said, we can use any of the following to get equivalent answers:

1. The standard standard dot-product:

In : a.dot(b.T)
Out: array([174, 174, 141, 119, 190])

2. The convenient numpy notation:

In : a @ b.T
Out: array([174, 174, 141, 119, 190])

3. The efficient "Einstein notation", a subset of Ricci calculus (I leave the interested reader to search online for more information):

In : np.einsum('i,ij->j', a, b.T)
Out: array([174, 174, 141, 119, 190])

4. Here as in the comments from shadowstalker:

In : np.array([np.dot(a, r) for r in b])
Out: array([174, 174, 141, 119, 190])

If your matrices are of dimensions (100, 100) or smaller, then the @ method is probably the fastest and most elegant. However, once you start getting into matrices that make you wonder if you laptop will handle it (e.g. with shape (10000, 10000)) - then it is time to read the documentation and this blog about Einstein notation and the amazing einsum module within numpy!

Ok, lets put this to a numpy test!

With Numpy, what’s the best way to compute the inner product of a vector of size 10 with each row in a matrix of size (5, 10)?

1. np.dot(vector1, matrix1)
2. np.array([np.dot(row, vector1) for row in matrix1])
3. np.matmul(vector1, matrix1)
4. np.dot(matrix1, vector1)
5. np.sum(matrix1 * vector1, axis=1)

In short,

The best answer is 4 and this is because it is the computationally cheapest method that takes a single step to complete.

But if you want to know why, then keep reading…

To answer this, lets see what works and what not. Remember the question asks for “the best way to computate the inner product”, so there should be more than one ways that works, right?

Now lets actually create a vector and a matrix with numpy

np.random.seed(2)

vector1 = np.random.randint(10, size=(1,10))
print('Vector 1\n', vector1)

matrix1 = np.random.randint(10, size=(5,10))
print('\nMatrix 1\n', matrix1)

We will get this output

Vector 1
[8 8 6 2 8 7 2 1 5 4]

Matrix 1
[[4 5 7 3 6 4 3 7 6 1]
[3 5 8 4 6 3 9 2 0 4]
[2 4 1 7 8 2 9 8 7 1]
[6 8 5 9 9 9 3 0 0 2]
[8 8 2 9 6 5 6 6 6 3]]

Awesome!

Now let’s play with each possible answer and see what fits and what not!

But before we do, let’s see how matrix multiplication works.

To multiply two matrices with [ROWS x COLUMNS] the columns of the first must match in size the rows of the second, so we need a scenario like the following.

[A,B] * [B,N]

The resulting matrix will have the dimensions of the outer dimensions of the two matrices So from the example above, we are allowed to multiply since the inner dimensions of the two matrices match, and we will gain a new matrix with the outer dimensions of the two matrices.

Our resulting matrix will be [A,N] dimensions in this case.

1. np.dot(vector1, matrix1)

OUTPUT: (EXCEPTION)
shapes (10,) and (5,10) not aligned: 10 (dim 0) != 5 (dim 0)

We have an **exception **here, so (1) cannot be correct. We cannot have a dot product since [1,10] * [5,10] don’t match the inner requirements for matrix multiplication.

2. np.array([np.dot(row, vector1) for row in matrix1])

OUTPUT:
[243 225 211 309 301]

Here it seems to work.

We have a “for” loop on our matrix for each row. So for 5 times, we will multiply each element of the matrix by the equivalent element of the vector and add them together.

On every run we will get a number, so after 5 times, we will have created a vector of size [1,5] with our product.

3. np.matmul(vector1, matrix1)

OUTPUT:
matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 5 is different from 10)

We have an exception here, so (3) cannot be correct.

Here we try to make matrix multiplication at [1,10]*[5,10]. The inner dimensions don’t match and we have the same problem as with (1).

4. np.dot(matrix1, vector1)

OUTPUT:
[243 225 211 309 301]

Here it seems to work.

We multiply a [5,10]*[10,1] matrix! The resulting matrix is a [5,1] matrix and gives the same result as with the case of answer (2).

However, this answer is better than the answer (2) since it is computationally cheaper, since in answer 2 we make a for loop, while here we get the result with a single algebraic calculation.

5. np.sum(matrix1 * vector1, axis=1)

OUTPUT:
[243 225 211 309 301]

Here it seems to work.

We get again the same result as with (2) and (4).

But again we do multiple steps here.

First by doing $$matrix1 * vector1$$ we end up with this matrix

[[32 40 42  6 48 28  6  7 30  4]
[24 40 48  8 48 21 18  2  0 16]
[16 32  6 14 64 14 18  8 35  4]
[48 64 30 18 72 63  6  0  0  8]
[64 64 12 18 48 35 12  6 30 12]]

Then by summing at axis 1 (y axis), we add each row numbers to a single number (much like the solution with the for loop in $$2$$).

So what is the BEST SOLUTION?

You remember that the question doesn’t just ask which of the options produces some result, but which one is best to use. So far answers $$2$$, $$4$$, and $$5$$ seemed to work.

The best answer is 4 and this is because it is the computationally cheapest method that takes a single step to complete.

RUN IT FOR YOURSELF!

import numpy as np

print("With Numpy, what’s the best way to compute the inner product of a vector of size 10 with each row in a matrix of size (5, 10)?")

np.random.seed(2)

print('\nSamples:')

vector1 = np.random.randint(10, size=(1,10))
print('\nVector 1\n', vector1)

matrix1 = np.random.randint(10, size=(5,10))
print('\nMatrix 1\n', matrix1)

#========================

print('\nA) np.dot(vector1, matrix1)', end='')
try: print(' - WORKING \n', np.dot(vector1, matrix1))
except Exception as e:   print(' - EXCEPTION \n', e)

#========================

print('\nB) np.array([np.dot(row, vector1) for row in matrix1])', end='')
try:
print(' - WORKING \n', np.array([np.dot(row, vector1) for row in matrix1]))
except Exception as e:
print(' - EXCEPTION \n', e)

#========================

print('\nC) np.matmul(vector1, matrix1)', end='')
try:
print(' - WORKING \n', np.matmul(vector1, matrix1))
except Exception as e:
print(' - EXCEPTION \n', e)

#========================

print('\nD) np.dot(matrix1, vector1)', end='')
try:
print(' - WORKING \n', np.dot(matrix1, vector1))
except Exception as e:
print(' - EXCEPTION \n', e)

#========================

print('\nE) np.sum(matrix1 * vector1, axis=1)', end='')
try:
print(' - WORKING \n', np.sum(matrix1 * vector1, axis=1))
except Exception as e:
print(' - EXCEPTION \n', e)

print('\n=======================================')

print("As the question asks, we shouldn't just look for a working result, but for the best result, and it seems to be (D) as it gets the best result with a single math step, thus computationally more efficient ")

As I know, fastest way to do it is:

np.dot(matrix, vector)

It is a function that specifically written and optimized for this. I don't recommend to multiply one by one and adding by yourself, obviously it will take more time.