# Criteria for assessing difficulty of a question

I have a list of questions and how many times they have been answered correctly and incorrectly. Based on this, I applied the formula:

score = (number of true answers)/(number of true answers + number of false answers)


which is a number between 0 and 1. The closer to 1, the easier the question.

Now, I have the average time (in miliseconds) children spent on a question. The maximum time is 853339.5 and the minimum time is 1254.8. The majority of values are, however, smaller than 20000, and there is barely anything over 40000.

I assumed that the longer the answering time, the more difficult the question and applied the following method:

Suppose the value is 1 if the time is smaller or equal than 2500. Then it drops by 20% with every 500 milliseconds that have passed. If it hits or is greater than 50000, it stays at 0.

The results are VERY inconsistent with the scores from the "true/false answers" method. I played around with the parameters and it's still bad. How can I obtain something more similar to it?

• I think any method based on time wouldn't measure the same thing as the proportion of correct answers, because spending more time on a question is not necessarily a sign that the question is harder. It depends what kind of question this is about but there are many possible biases, for example one can skip a question quickly if they don't know the answer. Jun 27 at 21:17