4
$\begingroup$

I was trying out various projects available for question generation on GitHub namely NQG,question-generation and a lot of others but I don't see good results form them either they have very bad question formation or the questions generated are off-topic most of the times, Where I found one project that actually generates good questions

bloomsburyai/question-generation

It basically accepts a context(paragraph) and an answer to generate the question and I am trying to validate the questions generated by passing the generated question along with the paragraph to allenNLP

Answer generation for a question

And then I am trying to make sure the generated answers are correct for the questions generated with calculating the sentence embedding for both the answers(AllenNLP and PotentialAnswer) using Universal Sentence Encoder and a cosine distance to get how similar the answers match and the filtering question that has least cosine distance.

Wanted to know if this is the best approach or Is there a state of the art implementation for question generation? Please suggest

$\endgroup$
  • $\begingroup$ The way you are doing will suffice if you are going to feed it to a neural network for training. But if you want to display it to user then it may not be that appealing. What is your actual purpose? $\endgroup$ – Sandeep Bhutani Aug 3 '19 at 18:03
  • $\begingroup$ @SandeepB my end purpose is to display it to the user. $\endgroup$ – Sundeep Pidugu Aug 6 '19 at 7:14
2
$\begingroup$

It seems that state of art methods use neural encoder-decoder models [1]

[1] Neural Question Generation from Text: A Preliminary Study : https://arxiv.org/pdf/1704.01792.pdf

There is an open source implementation of the paper written with Pytorch on github : https://github.com/magic282/NQG

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Is there any open-source implementation of the same? $\endgroup$ – Sundeep Pidugu Jul 29 '19 at 9:14
  • $\begingroup$ Yes, I edited my post $\endgroup$ – Alexis Pister Jul 29 '19 at 9:56
  • $\begingroup$ the results are not at all good :\ $\endgroup$ – Sundeep Pidugu Aug 2 '19 at 9:54
1
$\begingroup$

Keep an eye out on https://github.com/sebastianruder/NLP-progress/blob/master/english/question_answering.md which is regularly updated, the main repository https://github.com/sebastianruder/NLP-progress also shows the state-of-the-art on other tasks

|improve this answer|||||
$\endgroup$
  • 1
    $\begingroup$ Thanks but I see all of them are somehow related to question-answer generation but nothing specific to question generation as such but will be watching the repo for changes. $\endgroup$ – Sundeep Pidugu Aug 6 '19 at 7:15
  • $\begingroup$ @SundeepPidugu you could try running some of the models in reverse perhaps? Swapping the input and output, don't know if that's valid but both seem like seq2seq tasks $\endgroup$ – JoelKuiper Aug 6 '19 at 8:30
  • $\begingroup$ No, question answering models are completely different from question generation models since both the problems try to predict different structures. It won't be trivial to reverse it. $\endgroup$ – Caxton Dec 10 '19 at 13:24
1
$\begingroup$

For your first part of the question as to which question generation approaches are good - Neural question generation is being pretty popular (as of 2018/2019) among NLP enthusiasts but not all systems are great enough to be used directly in production. However, here are a few recent ones which reported the state-of-art performances in 2019 and have shared their codes too:

  1. https://github.com/ZhangShiyue/QGforQA
  2. https://github.com/PrekshaNema25/RefNet-QG

Besides, if you want more control as to understand and fix for wrongly generated questions, I would suggest the more traditional rule-based approach like the below which is more reliable than the above neural ones ones and generates a larger amount of question-answer pairs than the above 2:

  1. http://www.cs.cmu.edu/~ark/mheilman/questions/

To answer your second question, if your QG model is generating an answer, then it makes sense to use cosine similarity. Assuming your question generation is at the sentence level, you will mostly have short answer spans and hence averaging Glove or Paragram word vectors might serve you better results than the Universal Sentence Encoder.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.