In Tensorflow, I saw the following example:

import tensorflow as tf 
import numpy as np 

mat_a = tf.constant(np.arange(1,13, dtype=np.int32), shape=[2,2,3])  
mat_b = tf.constant(np.arange(12,24, dtype=np.int32), shape=[2,3,2])  
mul_c = tf.matmul(mat_a, mat_b)

with tf.Session() as sess:  
   runop = sess.run(mul_c)  

[[[ 88  94]  
  [214 229]]  
 [[484 508]  
  [642 674]]]

How does the tensor multiplication work?


3 Answers 3


You may want to read the documentation.

output[..., i, j] = sum_k (a[..., i, k] * b[..., k, j]), for all indices i, j.

For instance, in your example

$~~88=1\times12+2\times14+3\times16,~~~94=1\times13+2\times15+3\times17$ $214=4\times12+5\times14+6\times16,~229=4\times13+5\times15+6\times17$


I'll give you a small example, if you do the following Kronecker product \begin{equation} \begin{bmatrix} \color{red}{1} \\ \color{green}{5} \\ \color{blue}{10} \end{bmatrix} \otimes \begin{bmatrix} 2 \\ 4 \end{bmatrix} = \begin{bmatrix} \color{red}{1} \begin{bmatrix} 2 \\ 4 \end{bmatrix} \\\\ \color{green}{5} \begin{bmatrix} 2 \\ 4 \end{bmatrix} \\\\ \color{blue}{10} \begin{bmatrix} 2 \\ 4 \end{bmatrix} \\ \end{bmatrix} = \begin{bmatrix} 2 \\ 4 \\ 10 \\ 20 \\ 20 \\ 40 \end{bmatrix} \end{equation} The Kronecker product works the same way for matrices as well.


Tensor multiplication is just a generalization of matrix multiplication which is just a generalization of vector multiplication.

Matrix multiplication is defined as:

$$ A_i \cdot B_j = C_{i, j}$$

where $i$ is the $i^{th}$ row, $j$ is the $j^{th}$ column, and $\cdot$ is the dot product. Therefore it just a series of dot products.

One can then see how this extends to tensors: $$\mathbf{A}_{i} \cdot \mathbf{B}_{j} = \mathbf{C}_{i, j} $$

where $i$ is the $i^{th}$ row-wise matrix of the tensor, and $j$ is the $j^{th}$ column-wise matrix of the tensor... and is therefore just a series of matrix multiplications - or a series of a series of dot products.

Assuming all tensors are of rank three(it can be described with three coordinates):

$$\mathbf{A} \otimes \mathbf{B} = \mathbf{A}_{i, j} \cdot \mathbf{B}_{j, k} = \mathbf{C}_{i, j, k}$$

which means the $(i,j)^{th}$ vector of $\mathbf{A}$ times the $(j, k)^{th}$ vector of $\mathbf{B}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.