0
$\begingroup$

I'm creating a churn model and would like to create a ratio (# customers / total transaction) for each merchant. About 70% of the data are NaNs (zero/zero).

I was wondering what I should impute for the 70% of the NaNs. I have other features and I don't like to delete the 70% of the data.

But if I impute 0s, the distribution would probably become different from ground truth since lower number means large transaction volume with a few customer. If I impute mean though, it'll also be different since the element (zero / zero) has no actions fundamentally.

I was about to impute -1 to distinguish the NaNs with non-NaNs. Would that make sense for a feature of binary classifier?

Thanks!

$\endgroup$
  • $\begingroup$ Has anyone tried a different way to generate such features? Instead of customers per transactions, using <some scalar> + customer per transaction (with the NaNs imputed 0)? $\endgroup$ – Srikrishna Jul 20 '18 at 8:27
1
$\begingroup$

I would probably impute with a constant, 0 or -1 or something. However, I would though add another feature which would be a boolean flag to indicate if there has been any transactions or not. This gives extra help to the model in distinguishing between the cases of real values and imputed values.

| improve this answer | |
$\endgroup$
1
$\begingroup$

The usual solution is a mix of a few methods :

  1. remove these records, but this should be avoided if possible

  2. encode it based on general business knowledge, you could pick a distinct number ( extremely large or small ) and then train the model on it.

  3. finally, I like usually like to plot one-way charts and impute with values with the dependent (say churn rate) close enough to the observed dependent rate over the NaN values for that variable.

  4. Edit - more recently, I have been using the package MICE and think it could be used for imputing when you cannot do point 2 and there are far too many variables to do point 3 individually.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.