# How to deal with Missing Not at Random Data for k-means clustering?

I am running k-means clustering on a customer dataset. One of the available demographic fields is inferred homevalue, represented as an integer. This field has value 0 when it's inferred that the customer is not a homeowner at all (they are more likely a renter, live with relatives, etc). I'm struggling to think of a good way to treat this value.

Does it make sense to keep this value as 0? Then my understanding is that the algorithm will interpret this as someone who doesn't own property is closely related to someone who owns an extremely low value property which doesn't seem intuitively right.

Is there a better way of dealing with this?

• I think your question is not clear to me. What do you mean the value of homevalue is zero? What else it can be? Is it categorical then? It is best if you give a few lines of your dataset! Aug 11 '18 at 7:39
• Your attributes will likely have very different scale already. Try to think through what you are actually computing: does this make any sense for your application? What is the appropriate way of scaling the data, and why? Or are you just using k-means because it is the only thing you know? How can you know if a result is really good, or just "average"? Aug 11 '18 at 19:48

I think there are two approaches you can take:

# 1) Square the 'homevalue'

Assuming that homevalue is a numeric you can square this value.

Then, you will increase the difference between $0$-valued homes and low-valued homes. Further, it will make the difference between high-valued homes to low-valued homes also larger.

# 2) Split into two features

You can also split this information into two different features:

• Home value $v$, where $v \in \mathbb{R}^+$
• Owns a home $h$, where $h \in \{true, false\}$ and $h = \substack{false, \quad if \, v \, equals \, 0\\true, \quad \ otherwise}$

This way, your model has a way to explicitly distinguish between people that own or not a home.

Then hopefully you have another feature indicating the lifestyle or earnings of a person that doesn't own a home.

• Downvoter care to comment? Aug 12 '18 at 6:36
• I didn't downvote, but squaring doesn't seem motivated at all. Jun 10 '20 at 3:46
• Adding a flag 'helps' assuming you scale the inputs too, but you're still left with the problem that "0" is interpreted as a home value. I'd guess the best you can do is impute a mean value. Jun 10 '20 at 3:47

When you have a case of missing not at random, the best thing to do is create a new feature. You can add a new feature which has value 0 if the person doesn't own a house and 1 if he owns a house.

Also K-means is usually not used for clustering. You could go for other algorithms such as Hierarchical or DBScan.