In Tensorflow, I saw the following example:

import tensorflow as tf 
import numpy as np 

mat_a = tf.constant(np.arange(1,13, dtype=np.int32), shape=[2,2,3])  
mat_b = tf.constant(np.arange(12,24, dtype=np.int32), shape=[2,3,2])  
mul_c = tf.matmul(mat_a, mat_b)

with tf.Session() as sess:  
   runop = sess.run(mul_c)  

[[[ 88  94]  
  [214 229]]  
 [[484 508]  
  [642 674]]]

How does the tensor multiplication work?


You may want to read the documentation.

output[..., i, j] = sum_k (a[..., i, k] * b[..., k, j]), for all indices i, j.

For instance, in your example

$~~88=1\times12+2\times14+3\times16,~~~94=1\times13+2\times15+3\times17$ $214=4\times12+5\times14+6\times16,~229=4\times13+5\times15+6\times17$

| improve this answer | |

I'll give you a small example, if you do the following Kronecker product \begin{equation} \begin{bmatrix} \color{red}{1} \\ \color{green}{5} \\ \color{blue}{10} \end{bmatrix} \otimes \begin{bmatrix} 2 \\ 4 \end{bmatrix} = \begin{bmatrix} \color{red}{1} \begin{bmatrix} 2 \\ 4 \end{bmatrix} \\\\ \color{green}{5} \begin{bmatrix} 2 \\ 4 \end{bmatrix} \\\\ \color{blue}{10} \begin{bmatrix} 2 \\ 4 \end{bmatrix} \\ \end{bmatrix} = \begin{bmatrix} 2 \\ 4 \\ 10 \\ 20 \\ 20 \\ 40 \end{bmatrix} \end{equation} The Kronecker product works the same way for matrices as well.

| improve this answer | |

Tensor multiplication is just a generalization of matrix multiplication which is just a generalization of vector multiplication.

Matrix multiplication is defined as:

$$ A_i \cdot B_j = C_{i, j}$$

where $i$ is the $i^{th}$ row, $j$ is the $j^{th}$ column, and $\cdot$ is the dot product. Therefore it just a series of dot products.

One can then see how this extends to tensors: $$\mathbf{A}_{i} \cdot \mathbf{B}_{j} = \mathbf{C}_{i, j} $$

where $i$ is the $i^{th}$ row-wise matrix of the tensor, and $j$ is the $j^{th}$ column-wise matrix of the tensor... and is therefore just a series of matrix multiplications - or a series of a series of dot products.

Assuming all tensors are of rank three(it can be described with three coordinates):

$$\mathbf{A} \otimes \mathbf{B} = \mathbf{A}_{i, j} \cdot \mathbf{B}_{j, k} = \mathbf{C}_{i, j, k}$$

which means the $(i,j)^{th}$ vector of $\mathbf{A}$ times the $(j, k)^{th}$ vector of $\mathbf{B}$.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.