Number of parameters in Simple RNNs

Question

Please, I am stuck, I can not understand the number of parameters of a simple RNN, here the example and the model summary. the example is simple:

x = np.linspace(0,50,501)
y= np.sin(x)
df= pd.DataFrame(data=y, index=x, columns=['Sinus'])

Then I would to build a simple RNNs to predict this sine wave,

test_percent = 0.1
test_point= np.round(len(df)*test_percent)
test_ind = int(len(df)-test_point)
train = df.iloc[:test_ind]
test = df.iloc[test_ind:]

from sklearn.preprocessing import MinMaxScaler
scaled_train = scaler.transform(train)
scaled_test = scaler.transform(test)

from tensorflow.keras.preprocessing.sequence import TimeseriesGenerator
generator = TimeseriesGenerator(scaled_train, scaled_train, length=50, batch_size=1)

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, SimpleRNN

n_features=1
model= Sequential()
model.add(SimpleRNN(units=50, input_shape=(50,1)))
model.add(Dense(1))
model.compile(loss='mse', optimizer='adam')
model.summary()

I do not understand the number 2600 in the model summary! because according to this intuition of RNNS

in my case the dimension of U is (50,1), so 50 weights, then for V, also 50 weights, and for W, also 50 weights, for me, only 150 parameters, why this number in the summary: 2600 parameters?

Answer 1

Keras SimpleRNN is a fully connected RNN so indeed each unit connected with all other units. So the equation becomes:

• (input_feature +1) x units + units x units
• 2x50 +2500
• +1 comes from the bias