0
$\begingroup$

Lets assume that I have 5000 articles and I create the TF-IDF of these articles.

Now I ask som people to answer 30 questions and I create the TF-IDF of these answers from the IDF of the articles and use the cosinus similarity to find the 20 closest articles per person with a weight of 1 for each answer. People can mark these 20 articles as relevant to her/him or not relevant(0 or 1).

So, which approach would you follow to see which questions are the most important in their choices? In my mind it seems like feature selection problem. Or not? Any recommendation?

Generally, what I want to do is reduce the number of questions from 30 -> 5 and I have to find which one are the most important questions.

Edit

As @null_9 mentioned in a comment, I am not sure which is the best way of providing the most relevant articles: 1) Summing up the similarity scores from all the answers and give the highest, or 2) find N closest articles per each question and sample 20 of them and provide to the user.

$\endgroup$
4
  • $\begingroup$ The question seems confusing. Why do you want people to tag articles as good or bad? Are you giving only 20 suggestions combining all 30 questions? or each answer he gives gets 20 suggestions? Maybe give an example to help. $\endgroup$ Nov 9 '16 at 11:35
  • $\begingroup$ From the 30 questions 20 suggested articles will be given on total, for each person. And the person will evaluate the articles as relevant or not. $\endgroup$ Nov 9 '16 at 12:15
  • $\begingroup$ Of the 30 answers, will you find closest document to each answer (30 documents) and provide 20 closest ones, or sum up all the counts in 30 answers and find the closer 20 documents to the sum. ? $\endgroup$
    – chmodsss
    Nov 9 '16 at 13:05
  • $\begingroup$ Good question! Just became one of my additional concern. I was thinking of summing up. Which approach would you follow? $\endgroup$ Nov 9 '16 at 13:50
1
$\begingroup$

If selecting the most important and relative questions is the question and you choose the option:

option 1.) find the closest document to each answer(30 docs) and after the documents are validated relative or not. You will have the results of each person's evaluation (Xpersons x 30 scores). Now order the questions in descending order from most scored question to less scored one.

option 2.) Summing up all the answers and finding 20 docs. After the validation of relativeness by the persons, it is not possible to find the important features(questions) because they are summed up and the information is already lost. From this result you could only find which person validates closer to your model among all, which is not the aim of the work.

$\endgroup$
4
  • $\begingroup$ Why the information is lost in the second case? Even though the similarity scores are summed up, the scores are still there. $\endgroup$ Nov 9 '16 at 14:37
  • $\begingroup$ Yes, the scores are still there, but the individual scores for each questions are not there. So, the contribution of each question in the whole score cannot be obtained. $\endgroup$
    – chmodsss
    Nov 9 '16 at 14:40
  • $\begingroup$ Thats true. What I am thinking is to provide articles by summing coupling of questions (q1+q2, q1+q3 etc) to catch also compination of questions. What concerns me though is that this questionary is provided to small number of people. By your asnwer we cant prove the significance level to reject a question. Propably t-test could help on that. $\endgroup$ Nov 9 '16 at 14:49
  • $\begingroup$ if your features depend on perons who are answering, then ofcourse you need a larger group of people for survey. $\endgroup$
    – chmodsss
    Nov 9 '16 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.