thanks a lot for replying. appreciating it really as I'm confused on this point .. so the method which called get_bert_embed_matrix for getting word_embed_6 = torch.cat([hidden_states[i] for i in [-1,-2,-3,-4]], dim=-1) for sub-word and i need to calculate the mean for those values to get word-embedding

excuse me i got from your answer that BERT works on sub-words, not Word. so the method for concatenating the last four layers of BERT will represent sub-words ?

Thanks a lot if you have a good background in captioning can you please give me your email . Thanks in advance

Thakns. you mean like that model=Model(inputs=model.inputs,outputs=model.layers[-2].out‌​put)

Thanks a lot for replying , but excuse me if I want these features for captioning .. will I remove the last only ?

Did you solve it please

thanks a lot for replying .. can you please see the edit post