1
$\begingroup$

Please, I am stuck, I can not understand the number of parameters of a simple RNN, here the example and the model summary. the example is simple:

x = np.linspace(0,50,501)
y= np.sin(x)
df= pd.DataFrame(data=y, index=x, columns=['Sinus'])

Then I would to build a simple RNNs to predict this sine wave,

test_percent = 0.1
test_point= np.round(len(df)*test_percent)
test_ind = int(len(df)-test_point)
train = df.iloc[:test_ind]
test = df.iloc[test_ind:]

from sklearn.preprocessing import MinMaxScaler
scaled_train = scaler.transform(train)
scaled_test = scaler.transform(test)

from tensorflow.keras.preprocessing.sequence import TimeseriesGenerator
generator = TimeseriesGenerator(scaled_train, scaled_train, length=50, batch_size=1)

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, SimpleRNN

n_features=1
model= Sequential()
model.add(SimpleRNN(units=50, input_shape=(50,1)))
model.add(Dense(1))
model.compile(loss='mse', optimizer='adam')
model.summary()

The model Summary

I do not understand the number 2600 in the model summary! because according to this intuition of RNNS RNNs Model

in my case the dimension of U is (50,1), so 50 weights, then for V, also 50 weights, and for W, also 50 weights, for me, only 150 parameters, why this number in the summary: 2600 parameters?

$\endgroup$

1 Answer 1

2
$\begingroup$

Keras SimpleRNN is a fully connected RNN so indeed each unit connected with all other units. So the equation becomes:

  • (input_feature +1) x units + units x units
  • 2x50 +2500
  • +1 comes from the bias

enter image description here

$\endgroup$
1
  • 1
    $\begingroup$ Thank you very much for this clarification yes I see ! $\endgroup$ Jan 25, 2021 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.