Here's the scenario, There's a database with thousands of single-option questions for testing a specific skills, and a large number of users (either professional or amateur in this skill), each of which answer 10 random questions from the database.

The only thing that I can think of is to differentiate question difficulty level according to the correct rate of each question. But how could I take fully use of other informations like:

  1. the correct rate from the user's perspective and feedback to its own influence to the difficulty level of questions (if user A answers 9 out of 10 questions correct, then the incorrect one (question_10) is more likely to be harder, than an user B answers 1 out of 10 questions correct and question_10 is in the incorrect set)
  2. the answer time for each question by each user

Could anyone give me some ideas on this model, like where should I delve more into to make the difficulty level of question more accurate? Great thanks!


I suggest you look at item response theory and the Rasch model. They construct a model that attempts to identify both (a) the proficiency/knowledge/ability of the user and (b) the level of difficulty of the question.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.