How to detect the begin word and end word in a sentence with machine learning

Question

I have some English text that has been tokenized. For example, the length of text token is about 20000 and each word (tokenize) has an index. Also, each index has a label, as the beginning word in a sentence labeled as 'b', as an end word (include symbol) in the sentence labeled as 'e', with other words labeled as 'o'. There is labeled training data and unlabeled test data.

My question: how do we use a machine learning method or deep learning model to solve the issue that predicts the label of words in the test data? I mean how do we make the label and data for training just a word for training? I am confused.

Answer 1

For this kind of problem, you can study some of the latest research areas of reading comprehension in which people are trying to predict the starting index of the answer & with the help that index they are predicting ending index of the answer.

Following screenshot took from BiDaF research paper Refer to BiDaF for more details