0
$\begingroup$

I am trying to build a sequence to sequence model. I have used Vision transformer as the encoder and LSTM with 1 layers as the decoder. The output of the encoder is given as the hidden state for decoder LSTM and it tries to predict the caption. Is this way of doing image captioning wrong? The model is not working, tried tuning all the hyperparaeters and the hidden state size also.

Improve this question
$\endgroup$

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.