My task is to create a QA-model. I give it a context and a question that it should answer. The answer is usually one word, so a very simplified input would be e.g.

Context: "Max eats a banana. Now Max and Tim are going to the gym. While Max eats a banana on the way, Tim eats some chocolate."

Question: "What does Max eat?"

Answer: "(a) banana(.)"

I use a pretained model that predicts start and end tokens. To be more precise, it assigns a value to each of the tokens. Assuming each word would be tokenized into 1 token, I would have 2 vectors of length ~30, representing how likely it is that the answer to the question starts or ends at token x.

This is how the predictions for the start_tokens could look like: X = [10, 3, -4, 1, ..., 9, ...] where 10 and 9 are the index of the 2 relevant occurrences of "Max"

The training data is also labeled with those start and end indices (on char level, but I can easily transform it to token level). I thought of a comparison matrix like Y = [10, -10, -10, -10, ..., 10, ...], basically assigning all valid start indices the value 10 and all the rest -10.

This does not perform well, tho. I tried many different loss functions on this setup but everything fails. Any ideas on how to get this running? Am I maybe on the completely wrong track here with what I'm doing?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.