0
$\begingroup$

Lets assume that I have 5000 articles and I create the TF-IDF of these articles.

Now I ask som people to answer 30 questions and I create the TF-IDF of these answers from the IDF of the articles and use the cosinus similarity to find the 20 closest articles per person with a weight of 1 for each answer. People can mark these 20 articles as relevant to her/him or not relevant(0 or 1).

So, which approach would you follow to see which questions are the most important in their choices? In my mind it seems like feature selection problem. Or not? Any recommendation?

Generally, what I want to do is reduce the number of questions from 30 -> 5 and I have to find which one are the most important questions.

Edit

As @null_9 mentioned in a comment, I am not sure which is the best way of providing the most relevant articles: 1) Summing up the similarity scores from all the answers and give the highest, or 2) find N closest articles per each question and sample 20 of them and provide to the user.

Improve this question
$\endgroup$
4
  • $\begingroup$ The question seems confusing. Why do you want people to tag articles as good or bad? Are you giving only 20 suggestions combining all 30 questions? or each answer he gives gets 20 suggestions? Maybe give an example to help. $\endgroup$
    – Hima Varsha
    Commented Nov 9, 2016 at 11:35
  • $\begingroup$ From the 30 questions 20 suggested articles will be given on total, for each person. And the person will evaluate the articles as relevant or not. $\endgroup$
    – Mpizos Dimitris
    Commented Nov 9, 2016 at 12:15
  • $\begingroup$ Of the 30 answers, will you find closest document to each answer (30 documents) and provide 20 closest ones, or sum up all the counts in 30 answers and find the closer 20 documents to the sum. ? $\endgroup$
    – chmodsss
    Commented Nov 9, 2016 at 13:05
  • $\begingroup$ Good question! Just became one of my additional concern. I was thinking of summing up. Which approach would you follow? $\endgroup$
    – Mpizos Dimitris
    Commented Nov 9, 2016 at 13:50

1 Answer 1

Reset to default
1
$\begingroup$

If selecting the most important and relative questions is the question and you choose the option:

option 1.) find the closest document to each answer(30 docs) and after the documents are validated relative or not. You will have the results of each person's evaluation (Xpersons x 30 scores). Now order the questions in descending order from most scored question to less scored one.

option 2.) Summing up all the answers and finding 20 docs. After the validation of relativeness by the persons, it is not possible to find the important features(questions) because they are summed up and the information is already lost. From this result you could only find which person validates closer to your model among all, which is not the aim of the work.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Why the information is lost in the second case? Even though the similarity scores are summed up, the scores are still there. $\endgroup$
    – Mpizos Dimitris
    Commented Nov 9, 2016 at 14:37
  • $\begingroup$ Yes, the scores are still there, but the individual scores for each questions are not there. So, the contribution of each question in the whole score cannot be obtained. $\endgroup$
    – chmodsss
    Commented Nov 9, 2016 at 14:40
  • $\begingroup$ Thats true. What I am thinking is to provide articles by summing coupling of questions (q1+q2, q1+q3 etc) to catch also compination of questions. What concerns me though is that this questionary is provided to small number of people. By your asnwer we cant prove the significance level to reject a question. Propably t-test could help on that. $\endgroup$
    – Mpizos Dimitris
    Commented Nov 9, 2016 at 14:49
  • $\begingroup$ if your features depend on perons who are answering, then ofcourse you need a larger group of people for survey. $\endgroup$
    – chmodsss
    Commented Nov 9, 2016 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.