How to get vectors of memory cell & the last output of $LSTM$ in keras?

Question

In this research paper the following paragraph appears,

The state of every LSTM model is stored in two fixed-size vectors of real numbers called the memory cells and the last output. Since our LSTM model is trained to predict user’s behavior, elements of these vectors are the natural candidates for the user-dependent features. They can be extended by the resulting predictions (answers to the questions). This way 218 new features are obtained from the memory cells (100) and the last output (100) of the second LSTM layer and from the final output (18).

I am aware of getting weights of each layer. But how to get these two vectors?

Answer 1

Self answering after reading this article. Actually, the paragraph is saying that $218$ new features are obtained from -

1. Memory cells or Memory units of 2nd LSTM layer: It will give a vector of $100$ cell states (cell state corresponding to each memory cell). Note that the dimension of the vector is equal to the number of hidden memory cells.
2. Last output of 2nd LSTM layer: It will give a list of $100$ cell outputs (cell output of each memory cell/unit).
3. Final output: Final output would eventually comprise of an $18$ dimensional vector. Dimensions are - $2$ for (a), $6$ for (b), $4$ for (c), $3$ for (d), $2$ for (e), $1$ for (f).

From the same article, it can be achieved in keras using return_state=True parameter.

from keras.models import Model
from keras.layers import Input
from keras.layers import LSTM
from numpy import array
# define model
inputs1 = Input(shape=(3, 1))
lstm1, state_h, state_c = LSTM(2, return_sequences=True, return_state=True)(inputs1)
model = Model(inputs=inputs1, outputs=[lstm1, state_h, state_c])
# define input data
data = array([0.1, 0.2, 0.3]).reshape((1,3,1))
# make and show prediction
print(model.predict(data))

This ouputs,

[array([[[-0.00559822, -0.0127107 ],
    [-0.01547669, -0.03272599],
    [-0.02800457, -0.0555565 ]]], dtype=float32), array([[-0.02800457, -0.0555565 ]], dtype=float32), array([[-0.06466588, -0.12567174]], dtype=float32)]