# What is difference between Fully Connected layer and Bilinear layer in CNN?

What is the difference between Fully Connected layers and Bilinear layers in deep learning?

A bilinear function is a function of two inputs $x$ and $y$ that is linear in each input separately. Simple bilinear functions on vectors are the dot product or the element-wise product.
Let $M$ be a matrix. The function $f(x,y)=x^⊤My=∑_iM_ijx_iy_j$ is bilinear in $x$ and $y$. In fact, any scalar bilinear function on two vectors takes this form. Note that a bilinear function is a linear combination of $x_iy_j$ whereas a linear function such as $g(x,y)=Ax+By$ can only have $x_i$ or $y_i$. For neural nets, that means a bilinear function allows for richer interactions between inputs.
Now what if you want a bilinear function that outputs a vector? Well, you simply define a matrix $M_k$ for each coordinate of the output and you end up with a stack of matrices. That stack of matrices is called a tensor (3-mode tensor to be exact). You can imagine the bilinear tensor product with two vectors as $x^⊤M_ky$ computed on each “slice” of the tensor.