Let's say in an NLP problem, I have a question and some correct answers to that question (say, 10 correct answers).
Is there a way to get a new answer as input, and "calculate" whether it is correct?
$\begingroup$
$\endgroup$
-
$\begingroup$ From a noobs point of view, a correct answer can be depending on the words, the length of the answer etc, if we can restrict the length, and do some preprocessing and then let's say calculate the cosine similarity with the correct ones, we might have one... Again I am not sure whether this this will work or not, just a pipeline which came in my mind $\endgroup$ – Aditya Sep 10 '18 at 9:08
$\begingroup$
$\endgroup$
This problem can be solved better if you also include the background text from where the questions have been picked up. Then, firstly train word embeddings on the background text. Further, generate sentence compositionality of the questions as well as their answers using a Recursive Neural Network or some variant. After all these steps you can compute the similarity between compositions of the new answer and given answers to know if it is correct or not.
$\begingroup$
$\endgroup$
One approach to consider:
You will need a similarity measure (say cosine similarity), S(A,B) between two pieces of text, and a way to threshold this measure.
- For a specific answer A_i, compute the average of similarity scores S(A_i, A_j) between A_i, and all other answers A_j. Call this Sim_i.
- Compute Sim_i, for all answers A_i.
- Take the least of all Sim_i, as the threshold similarity, Sim_T
- Given a new answer A_new, compute its score, Sim_new with respect to all answers.
- If Sim_new >= Sim_T, then accept the answer, else reject it.
The idea is that the new answer should be accepted if it is "similar" to other known answers. How similar should it be? This is estimated using the similarity between known answers. This is the broad approach. The similarity measure and threshold computation can be experimented with.
