Machine Learning, Imputing values that should be blank

Question

Sometimes data sets contain variables that indicate the presence of an event and the value that represented the event.

As an example say a teacher wants to predict the grades of his students. Some of the students may have been in his class last year and he can use that grade as a variable. However maybe only 20% of the students were in his class so the rest of the 80% will have a Null value. Most ML algorithms cannot accept Null values so the variable would have to somehow be imputed.

I cannot think of an imputation method that would make sense here, the standard mean/mode would imply that all students were in the class and since the variable is pretty unbalance and 80% of the values would be imputed I don't imagine it would hold any valuable information.

Are there any methods to deal with this scenario?

Answer 1

Well, it seems that you are dealing with sparse data, however imputation is a difficult and often an attempt of imputation can add trivial amount of difference. You may look out on for this link for some approaches like Gharamani and Jordan.
These are variants of SVM, focused with Sparse nature.

Answer 2

Since last year's grade should be an important feature, we should use it whenever it is available.
I think that a stratified model should work here. Create 2 different models, one for last year students, the other for the remaining 80%. I think maybe there will be other features for the 20% sample, all the others will be in common for the 2 models.

Answer 3

For the specific case of the notes, you could try to transform it by categories, where the null values will have a different category.

Another option would be to impute by the mean or the median, but previously it would create a binary variable to identify the null values.