This blog covers the basics of LSTMs.
A forget gate is defined as :
$$f_t = \sigma(W_f \cdot [h_{t-1}, x_t]+ b_f)$$
At this point the linear algebra confuses me more than it should. The syntax of $W\cdot [h,x]$ is confusing in this context. I think a vector should go into the activation function since the output $f$ is a vector, but the syntax of the forget gate above implies that the input has $2$ columns because $[h,x]$ will be an $n\times 2$ matrix
For the sake of example lets say ...
\begin{align} W &= \begin{bmatrix} 0 & 1 \\ 2 &3 \end{bmatrix}\\ h &= \begin{bmatrix} -1 \\ 2 \end{bmatrix}\\ x &= \begin{bmatrix} 3 \\ 0 \end{bmatrix}\\ b &= \begin{bmatrix} 1 \\ -2 \end{bmatrix}\end{align}
Can anyone give the final vector that goes into the sigmoid function ?
I think the math is
$$ \begin{bmatrix} 0 & 1 \\ 2 & 3 \end{bmatrix}\begin{bmatrix} -3 & 3 \\ 2 & 0 \end{bmatrix} + \begin{bmatrix} 1 \\ -2\end{bmatrix} = \begin{bmatrix} 2 & 0 \\ 4 & 6\end{bmatrix}+ \begin{bmatrix} 1 \\ -2\end{bmatrix} = \text{ Something wrong}$$