- Is this correct?
Yes the diagrams both look correct to me. The key thing to understanding both diagrams is that the inputs and outputs of an LSTM cell are vectors.
The circles in the first diagram do represent the concept that the layer contains multiple individual artificial neurons, and that might make you assume that the second diagram is a picture of one of those neurons. Arguably there are multiple "neurons" or sub-layers with different roles inside a cell, because there are multiple places in which calculations of the form $f(W\mathbf{x} +b)$ occur, performing slightly different roles. I think the term "cell" is used to refer to this architecture of neurons. As a short-hand when I say "neuron", I tend to think of the output stage of the hidden layer.
However, in the second diagram, all of the operations shown work with vectors. Most importantly the left-to-right arrows in the second diagram represent vectors of hidden state from timestep to timestep from the whole layer. So each neuron in the cell is recurrently connected to every other neuron in that cell - twice as it happens in an LSTM, because LSTMs have both an internal cell state and a layer output.
- Is each unit in the cell independent from the others? Or do they share information?
To match your description of the diagrams, let's define a "unit" as a collection of one of each type of neuron/gate used to make up the cell, that in theory could be wired together to make a working LSTM cell layer with a single scalar cell state and output value.
These units are independent in that each has its own weight parameters. There are no shared parameters for the connections between input and the units, or for the recursive connections that forward state from one time step to the next. In that sense the units do not share information.
However, the connections do mean that on each time step, input data and hidden state plus output from last output from all other units in the cell are combined are used in calculations. Any cell unit can base its new internal state plus its output on the values of all other outputs and internal states from other units in the cell. In this sense, the units do share information. I guess from your question that it is probably this second issue that you are concerned about, as the second diagram makes you think of a wiring diagram for a single neuron, but as explained above that is not the case.
- Imagine I have the following configuration: Number of samples = 1000 Number of time-steps = 10 Number of features = 5
In this case, each unit in a cell will take as input a vector of size 5 right?
Almost. Each neuron inside the cell will take an input of 5 from $\mathbf{x}$, plus an input of the hidden layer output, $\mathbf{h}$. So if in your case the LSTM cell size was 10, then each neuron would take a combined vector of 15. In addition, a second cell state vector is maintained, not labelled in your diagram. That is not used directly as an input to any neurons (i.e. components of the form $f(W\mathbf{x}+b)$), but does interact with the other values and can get changed itself, through the various gates. In your second diagram, it is the uppermost arrow going left to right.
But what will be the size of the output for one unit? 1?
The cell as a whole will have the output of whatever size you have made the layer. That is what the diagram number 2 tries to show. However, using our working definition of "unit", the output of each unit will be two scalar values - the hidden layer output, and the cell state - which will be part of their respective vectors shown in the diagrams.
