# What is the difference between a "cell" and a "layer" within neural networks?

So I understand what "layers" are. If you have 5 layers in your model, your data basically gets transformed 5 times via 5 activation functions. The number of "neurons" within a layer dictate how many outputs a layer creates.

So what are "cells"? I never understood where "cells" come into play. Are they a collection of layers?

If the orange are layers, then I would imagine each has a bunch of neurons. So a cell is a collection of layers and yellow stuff? I'm having trouble understanding where this "cell" fits into an overall NN architecture. I am used to the pictures with input layer -> hidden layer -> output layer. So where would the "cell" occur?

• "Cell" is probably identical to "neuron," but it will be easier to answer accurately if you can point us to a use of "cell" in a paper/book/website etc. Aug 4 '20 at 19:12
• Well the book "deeplearning" by goodfellow starts talking about "cells" the second we start talking about RNNs and LTSMs. I am not understanding that makes a cell distinct from layers and nuerons. Also wikipedia uses "cell" as well when it talks about LTSM en.wikipedia.org/wiki/Long_short-term_memory Aug 4 '20 at 20:51
• Thanks for clarifying. Turns out out cell != neuron for your purposes. Aug 5 '20 at 20:34

### In short

A cell is in essence a forward neural network consisting of neurons and so $$cell_{RNN}\neq neuron_{NN}$$.

### Rationale

In the context of recurrent neural networks, a layer consists of cells e.g. LSTM cells. Although they seem to position similarly in terms of architecture (figure), there is a fundamental difference between an RNN_cell and a NN_neuron.

1. The number of RNN cells is generally equal to the sequence length (e.g. number of words in a sentence) or embedding output dimension (figure below). Conversely, the number of neurons in forward neural networks can generally be of any number.
2. Each RNN_cell acts much like a NN itself. Specifically in LSTMs, each cell consists of hidden state $$W_{hidden}$$ and cell state $$W_{cell}$$ matrices. These are used to store weights and perform transformations in the various gates (forget, input, output) of the LSTM cell.

In essence, the transformations that take place in each cell are matrix multiplications of $$W_{hidden}$$ and $$W_{cell}$$ much like those in a forward NN. And so, the size of $$W_{hidden}$$ and $$W_{cell}$$ must be equal to the number of neurons of that NN which essentially determine the memory of the LSTM cells and model itself.

• Thanks! I can't upvote you but otherwise I would. So a cell basically is a NN by itself. This would make sense as in my pic, there are layers each with their own activation functions. Aug 6 '20 at 19:36
• Happy to help. Yes in the context of sequence networks that is the case. In Keras LSTM you can define the size of the “NN” of each cell using the parameter units, a parameter used also in Dense layer that defines the size of a hidden layer dimension of a conventional forward NNs for example. Nov 13 '20 at 17:59

If you have 5 layers in your model, your data basically gets transformed 5 times via 5 activation functions.

That is not completely true. For dense neural networks (for example), your layers are made by neurons that each has its own activation function (so you'll have more than 5 activation functions). Each neuron transforms your data.

Now what is a cell ?

A cell is a mathematical function that receives one or more input, does the math, and gives one or more outputs.

First some examples:

1. neuron: the result is the output of an activation function applied to the weighted sum of the previous layer
2. convolution cell: passes the result of a convolution kernel multiplied to a relative range of the cells of the immediately previous layer to the next layer
3. a lot of other cell examples in this post

Why are they called "cells" ? Because those mathematical function are used in artifical neural networks (ANN) context.

Why do they have a name ? Because they work well in (some) ANN, so people share their new mathematical function that helps an ANN to converge given a specific problem.