Our product has an onboarding questionnaire which asks the same 58 questions (with numeric answers) to every new user. That’s a lot of questions, so we’d love to reduce the number of questions we ask each new user.
We figure that instead of asking all of the questions every time, we could create a system that asks the most “informative” question at each point, given all of the user’s previous responses so far - i.e. the question that does the most to improve the accuracy of our prediction about how they would answer the remaining questions.
I visualise this as a multiway tree, where each node represents a question and each branch represents a range of answers to that question. Every user sees the question at the root node, then each subsequent answer they give defines how they traverse the tree.
The questions finish when they reach a leaf node (e.g. at depth 7), but even if they quit the onboarding process early, the questions they have been asked should still provide the best-possible prediction of how they would answer all of the remaining questions from the 58.
The question is, how do we construct this tree? We have data from 348 customers who answered all 58 questions, so it should be possible - but what is the best algorithm?
We tried the RandomForestRegressor from SciKit, but that doesn’t seem at all suitable for this problem, as the trees are not at all in this format. Like other algorithms we’ve looked at, it’s great for predicting based on their previous answers - but not for knowing which questions to ask.
Can anybody suggest an algorithm (or class of algorithms) that is able to construct this tree?