How can we shorten our questionnaire to only ask the most informative question at each point?

Our product has an onboarding questionnaire which asks the same 58 questions (with numeric answers) to every new user. That’s a lot of questions, so we’d love to reduce the number of questions we ask each new user.

We figure that instead of asking all of the questions every time, we could create a system that asks the most “informative” question at each point, given all of the user’s previous responses so far - i.e. the question that does the most to improve the accuracy of our prediction about how they would answer the remaining questions.

I visualise this as a multiway tree, where each node represents a question and each branch represents a range of answers to that question. Every user sees the question at the root node, then each subsequent answer they give defines how they traverse the tree.

The questions finish when they reach a leaf node (e.g. at depth 7), but even if they quit the onboarding process early, the questions they have been asked should still provide the best-possible prediction of how they would answer all of the remaining questions from the 58.

The question is, how do we construct this tree? We have data from 348 customers who answered all 58 questions, so it should be possible - but what is the best algorithm?

We tried the RandomForestRegressor from SciKit, but that doesn’t seem at all suitable for this problem, as the trees are not at all in this format. Like other algorithms we’ve looked at, it’s great for predicting based on their previous answers - but not for knowing which questions to ask.

Can anybody suggest an algorithm (or class of algorithms) that is able to construct this tree?

• Hm, this gets a little trickier if you don't know how many questions they'll answer in total. You might find that there's one question that would let you predict the other 57 reasonably well, but that a different two questions would let you predict the other 56 even better. In this case, you'd ask different questions if you know they're only going to answer 1 or 2. I guess you'd need to take a greedy approach to select the next question sequentially, since you have to assume that might be the last question they answer. Jul 15 '21 at 14:19