8
$\begingroup$

I was trying out various projects available for question generation on GitHub namely NQG,question-generation and a lot of others but I don't see good results form them either they have very bad question formation or the questions generated are off-topic most of the times, Where I found one project that actually generates good questions

bloomsburyai/question-generation

It basically accepts a context(paragraph) and an answer to generate the question and I am trying to validate the questions generated by passing the generated question along with the paragraph to allenNLP

Answer generation for a question

And then I am trying to make sure the generated answers are correct for the questions generated with calculating the sentence embedding for both the answers(AllenNLP and PotentialAnswer) using Universal Sentence Encoder and a cosine distance to get how similar the answers match and the filtering question that has least cosine distance.

Wanted to know if this is the best approach or Is there a state of the art implementation for question generation? Please suggest

Improve this question
$\endgroup$
2
  • $\begingroup$ The way you are doing will suffice if you are going to feed it to a neural network for training. But if you want to display it to user then it may not be that appealing. What is your actual purpose? $\endgroup$
    – Sandeep Bhutani
    Commented Aug 3, 2019 at 18:03
  • $\begingroup$ @SandeepB my end purpose is to display it to the user. $\endgroup$
    – Spark
    Commented Aug 6, 2019 at 7:14

5 Answers 5

Reset to default
3
$\begingroup$

For your first part of the question as to which question generation approaches are good - Neural question generation is being pretty popular (as of 2018/2019) among NLP enthusiasts but not all systems are great enough to be used directly in production. However, here are a few recent ones which reported the state-of-art performances in 2019 and have shared their codes too:

  1. https://github.com/ZhangShiyue/QGforQA
  2. https://github.com/PrekshaNema25/RefNet-QG

This one is a 2020 one (now that NLP performances have been improved with Transformers) 3. https://github.com/patil-suraj/question_generation

Besides, if you want more control as to understand and fix for wrongly generated questions, I would suggest the more traditional rule-based approach like the below which is more reliable than the above neural ones ones and generates a larger amount of question-answer pairs than the above 2:

  1. http://www.cs.cmu.edu/~ark/mheilman/questions/
  2. https://bitbucket.org/kaustubhdhole/syn-qg/src/master/

To answer your second question, if your QG model is generating an answer, then it makes sense to use cosine similarity. Assuming your question generation is at the sentence level, you will mostly have short answer spans and hence averaging Glove or Paragram word vectors might serve you better results than the Universal Sentence Encoder.

Improve this answer
$\endgroup$
2
$\begingroup$

It seems that state of art methods use neural encoder-decoder models [1]

[1] Neural Question Generation from Text: A Preliminary Study : https://arxiv.org/pdf/1704.01792.pdf

There is an open source implementation of the paper written with Pytorch on github : https://github.com/magic282/NQG

Improve this answer
$\endgroup$
3
  • $\begingroup$ Is there any open-source implementation of the same? $\endgroup$
    – Spark
    Commented Jul 29, 2019 at 9:14
  • $\begingroup$ Yes, I edited my post $\endgroup$
    – Alexis Pister
    Commented Jul 29, 2019 at 9:56
  • $\begingroup$ the results are not at all good :\ $\endgroup$
    – Spark
    Commented Aug 2, 2019 at 9:54
0
$\begingroup$

Keep an eye out on https://github.com/sebastianruder/NLP-progress/blob/master/english/question_answering.md which is regularly updated, the main repository https://github.com/sebastianruder/NLP-progress also shows the state-of-the-art on other tasks

Improve this answer
$\endgroup$
3
  • 3
    $\begingroup$ Thanks but I see all of them are somehow related to question-answer generation but nothing specific to question generation as such but will be watching the repo for changes. $\endgroup$
    – Spark
    Commented Aug 6, 2019 at 7:15
  • $\begingroup$ @SundeepPidugu you could try running some of the models in reverse perhaps? Swapping the input and output, don't know if that's valid but both seem like seq2seq tasks $\endgroup$
    – JoelKuiper
    Commented Aug 6, 2019 at 8:30
  • 1
    $\begingroup$ No, question answering models are completely different from question generation models since both the problems try to predict different structures. It won't be trivial to reverse it. $\endgroup$
    – Caxton
    Commented Dec 10, 2019 at 13:24
0
$\begingroup$

To answer your question, I would say transformer based T5 models have performed state of the art in this particular problem statement i.e. Question Generation. Do give a try to this awesome repo: https://github.com/patil-suraj/question_generation. This guy is a regular contributor in HuggingFace.

Improve this answer
$\endgroup$
0
$\begingroup$

I have had great success with Haystack. Haystack provides state-of-the-art T5 models with many features such as Retriever, Reader, Generator, preprocessing, etc to simplify many NLP processes. Moreover, they also have the option of using pipelines which in turn use DAG (directed Acyclic Graphs) that determine how to route the output of one component into the input of another. Their tutorials are really good. Check out them!

Disclaimer: I am in no way affiliated with Haystack, just a dude using their OSS.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.