I've been exploring Closed Domain Question Answering Implementations which have been trained on SQuAD 2.0 dataset. Ideally, it should not answer questions which the context text corpus doesn't contain answers to. But while implementing such models using the Haystack repo or the FARM repo, I'm finding that it always answers these questions even when it shouldn't. Is there any implementation available that takes into account the fact that it shouldn't answer questions when it doesn't find a suitable answer.


  1. https://colab.research.google.com/github/deepset-ai/haystack/blob/update-tutorials/tutorials/Tutorial3_Basic_QA_Pipeline_without_Elasticsearch.ipynb#scrollTo=KS4nTwxIbRb6

  2. https://github.com/deepset-ai/FARM

  3. https://github.com/deepset-ai/haystack

  4. https://huggingface.co/deepset/bert-large-uncased-whole-word-masking-squad2

  5. https://colab.research.google.com/drive/1UrKlHlf68hD3wwQDTctx2cQs6FMUxLLH?usp=sharing#scrollTo=J4jxYsxaG77O

  • $\begingroup$ did you try my suggestion? $\endgroup$ – Ilker Kurtulus Jul 3 '20 at 13:57
  • $\begingroup$ @IlkerKurtulus yes I did, I also found a scalable implementation of the same from Haystack library. Thank You! $\endgroup$ – Anirban Saha Jul 6 '20 at 4:53
  • $\begingroup$ You are welcome. $\endgroup$ – Ilker Kurtulus Jul 6 '20 at 7:45

Question Answering model of simpletransformers takes data with is_impossible option in the training phase. Also during prediciton, it does not generate answers for all questions since the model learns which questions should be answered.

You can find details below:



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.