When reading papers on neural networks, I occasionally stumble upon the following notation with a semicolon:

$$ \text{tanh}(\mathbf{W_c}[\mathbf{c}_t;\mathbf{h}_t]) $$

Unless otherwise noted, does this by default mean the following:

  • vector $\mathbf{c}_t$ is appended to vector $\mathbf{h}_t$
  • the resulting long vector is dot-producted with a Weight matrix $\mathbf{W}_t$
  • Finally, the resulting vector is component-wise activated by a hyperbolic tangent function

The first bullet point is my main question. Googling for "Vector concatenation notation" doesn't return answers that would resemble the image I've attached above. However, many papers seem to use it.

An example of notation that I googled

  • 2
    $\begingroup$ Yes. Concatenation is the richest aggregation of mathematical objects. As opposed to, say, averaging. $\endgroup$
    – Emre
    Commented Apr 13, 2018 at 16:17


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.