I am confused about the parameters: $U^i, U^f, U^o , U^g$ and $W^i, W^f, W^o , W^g$. Are those parameters trained by data(e.g., back-propagation)?
Yes. They do get trained by backpropagation. There are caveats here. Variants to backprop too exist when training a sequence-to-sequence to models. Check out BPTT for example.
My second question is that, In the Figure, We have the three block of LSTM
3 (mark by blue color). Do them share the same $U^i, U^f, U^o , U^g$ and $W^i, W^f, W^o , W^g$. ? Or each block have their own $U^*$ and $W^*$?
Each block has the same set of weights shared among them.
An following mini example demo would be like in Keras.
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.layers import Embedding
from keras.layers import LSTM
model = Sequential()
model.add(LSTM(128, input_shape=(None, 100)))
creates an simple
LSTM model. If you look at model summary to count the params you got.
You will get
Layer (type) Output Shape Param #
lstm_3 (LSTM) (None, 128) 117248
dropout_2 (Dropout) (None, 128) 0
dense_2 (Dense) (None, 1) 129
Total params: 117,377
Trainable params: 117,377
Non-trainable params: 0
And the total of
117,377 params you see is obtained from
(((128*128)+(128*100)) * 4) + (4*128). Here W is of size
128*100 and U is
128*128 and there are
4 of them resp. and last
4*128 goes for the biases.