Suppose I have many vectors which can take on any of three values, 0, 1, 2. These vectors affect an outcome being predicted, Y. Vectors add together: a vector "A" of the value 2 has twice the affect on Y as vector "A" of the value 1, and also vector "A" and other vector "B" add together (that is, there are never interactions between vectors). Some vectors are very correlated with a small number of related vectors, and the degree to which they are correlated is known precisely.

Here is a difficulty: there is a very large number of vectors (features), about 1 million. However, there are only about 100,000 or so feature set --> outcome observations available for training, and there are 10,000 "true" (non-zero) features. Different features have different effect sizes.

What is the optimal machine learning technique to predict Y? I've been told LASSO would be a good choice but I want to hear other methods.

This question is crossposted here, as I'm not sure where this goes. I will update threads will be updated as soon as one gets a result. Feel free to ask me to delete one or the other if you know where this question should go.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.