2
$\begingroup$

I have ~9k training samples, the same pair vectors are labeled as 0 and not same pair samples are labeled as 1.

I trained it for 100 epochs for the first 3 epochs the loss value is fluctuating, then it returns the same loss value until the end.

Base network architecture:

def create_base_network():

    a = 'tanh'
    model = Sequential()
    model.add(Dense(self.INPUT_DIM, input_shape=(self.INPUT_DIM, ), activation=a))
    model.add(Dense(600, activation=a))
    model.add(Dense(600, activation=a))
    model.add(Dense(900, activation=a))
    model.add(Dense(1000, activation=a))
    model.add(Dense(5000, activation=a))
    model.add(Dense(1000, activation=a))
    model.add(Dense(900, activation=a))
    model.add(Dense(600, activation=a))
    model.add(Dense(600, activation=a))
    model.add(Dense(self.INPUT_DIM, activation=a))
    return model

Euclidean distance:

K.sqrt(K.maximum(K.sum(K.square(x - y), axis=0, keepdims=True), K.epsilon()))

Initilization model:

    base_network = self._create_base_network()
    input_a = Input(shape=(self.INPUT_DIM,))
    input_b = Input(shape=(self.INPUT_DIM,))

    processed_a = base_network(input_a)
    processed_b = base_network(input_b)

    distance = Lambda(self._euclidean_distance, output_shape=self._dist_output_shape)([processed_a, processed_b])
    model = Model(inputs=[input_a, input_b], outputs=distance)

Compilation of model:

model.compile(loss='mse',
          metrics=['mse'],
          optimizer=optimizers.Adam()
          )

Training the model:

model.fit([self.train_d['vec1'], self.train_d['vec2']], self.train_d['label'], batch_size=128, epochs=self.args.epochs, shuffle=True)

Log statements:

Epoch 1/100
9544/9544 [==============================] - 6s 594us/step - loss: 0.4556 - mean_squared_error: 0.4556
Epoch 2/100
9544/9544 [==============================] - 4s 470us/step - loss: 1.0693 - mean_squared_error: 1.0693
Epoch 3/100
9544/9544 [==============================] - 4s 464us/step - loss: 0.7328 - mean_squared_error: 0.7328
Epoch 4/100
9544/9544 [==============================] - 4s 465us/step - loss: 0.7328 - mean_squared_error: 0.7328
Epoch 5/100
9544/9544 [==============================] - 4s 461us/step - loss: 0.7328 - mean_squared_error: 0.7328
Epoch 99/100
9544/9544 [==============================] - 4s 470us/step - loss: 0.7328 - mean_squared_error: 0.7328
Epoch 100/100
9544/9544 [==============================] - 4s 470us/step - loss: 0.7328 - mean_squared_error: 0.7328

Can someone help me to find out what's the issue here?

Thanks in Advance.

$\endgroup$
1
$\begingroup$

Finally, I found a solution to this problem. I completely modified the architecture, Then I reduced the learning rate to 0.00005. Then the loss value of the model falls continuously.

Latest Architecture:

a = 'tanh'
model = Sequential()
model.add(Dense(self.INPUT_DIM, input_shape=(self.INPUT_DIM, ), 
activation=a))
model.add(Dense(600, activation=a))
model.add(Dense(self.INPUT_DIM, activation=a))
$\endgroup$
1
  • 2
    $\begingroup$ One more thing we need to change the axis value to 1, in the Euclidean distance formula. $\endgroup$ – chatrapathi Jun 29 '18 at 10:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.