4
$\begingroup$

I want to compute the "distance" between users in order to return the top n similar users, for any given user. For each user a have a bunch of features.

This is close to a recommendation system, however I don't have ratings and I can't assign each user to different products, hence I'll need to compare "how similar" Users are.

Do you have any suggestion for the algorithms/approaches for this kind of problem? I find a lot of resources based on recommendation system (having rating and items), but that's different from what I want to achieve

$\endgroup$
1
+50
$\begingroup$

As others also pointed out as long as you have numeric data (or data that can be converted to numeric) you can use some sort of distance measure between the users.

Simple solution is the euclidean distance (or some other like minkowski or manhattan). The gotcha with them that they are sensitive for having different scales in your variables. You can solve it by normalizing your data but keep in mind that in this case you will assign equal importance for each feature. You may want to adjust it manually based on your domain knowledge.

If you have a high number of features in a sparse space it's worth considering using cosine similarity that will more focus on the direction of your data point (customer features).

You may also want to do PCA before thereby reducing your dimensionality and eliminating the fact that certain features can be similar to each other (so they correlate).

If you want to experiment with a more sophisticated solution you can try to do an autoencoder. A type of neural network where your input and output are the same (user features) but in the hidden layers you suppress the dimensionality thereby having a more dense representation of the data. In that representation features may also bear a semantic meaning. On this more dense representation you can calculate again some of the distance metrics proposed at the beginning.

$\endgroup$
  • $\begingroup$ do you have any example of autoencoders applied to a similar problem? $\endgroup$ – Alex Jun 21 at 18:52
  • $\begingroup$ It's not perfectly the same but in some way word2vec embeddings are created in a similar fashion. You use the hidden layer representation of the words. Making simple vector calculations with them works surprisingly well. I also used a similar but more complex solution for image similarity. $\endgroup$ – Viktor Jun 21 at 19:03
  • $\begingroup$ my understanding of word2Vec is that it captures the "context" using neighborhood words when the dictionary is huge. In my case I have 60 features, can I still use the same approach? $\endgroup$ – Alex Jun 21 at 19:07
  • $\begingroup$ One (big) advantage of word2vec is that they significantly reduce the dimensionality but the other one is that the new features will have a semantic meaning. It is possible that you have even the same number of features as before but they are more meaningful. I think you can test the same approach but if it will work or not you will see it from the results. (as it is the case with all ML problems:)) $\endgroup$ – Viktor Jun 21 at 19:13
4
$\begingroup$

No need for algorithms, or recommendation systems. You have:

For each user a have a bunch of features.

As long as they're numeric, or can be made numeric (e.g. aggregating the values or one-hot-encoding them), you already have distances. What you may not have is the proper variance across the feature space, i.e. features are scaled in different orders of magnitude.

If you know the exact weight of the features in relation to user similarity you may try to tune (scale) the features by hand. Otherwise you can simply make every feature have mean 0 and standard deviation 1. In other words, per feature subtract the mean from all points and divide by the current standard deviation. (sklearn has a StandardScaler that does exactly that.)

In the scaled dataset, from any point (user), you can just calculate euclidean distance to any other point. And the closest the points the more similar the pair of user will be. i.e. top $N$ similar users to a user are just the $N$ closest points.

Plain euclidean distance works in many cases. If euclidean distance does not work for the problem at hand, then you can explore more complex possibilities: starting from manhattan distance, through minkowski distance (combination of euclidean and manhattan distances).

$\endgroup$
  • $\begingroup$ Thanks! I'm going to try this approach - do you have any source I can look up? (I'm trying to visualize this with Tensorflow ) $\endgroup$ – Alex Jun 18 at 1:22
  • $\begingroup$ @Alex - TensorFlow is for ANNs, they're poor visualization tools. For a handful of features (e.g. up to 4-5) just scaling and visualizing on a graph (or couple of histograms) should be enough. For a lot of features I'd go with visualization techniques, t-SNE would be my first try to visualize a lot of features at once. If t-SNE results in well separated groups (and number of groups you expect) then distances are a good estimate of similarity. $\endgroup$ – grochmal Jun 20 at 22:29
1
$\begingroup$

The main thing one thinks about is some measure of distance. Treating each variable as an axis, users can be reprsented as datapoints in a multidimensional space. Euclidean distance is the most common, but you can use Manhattan, Minkowsky, Mahalanobis, ... there are countless formulas.

Common alternatives are (dis)similarity measures, such as cosine similarity and KL divergence. They all return measures of "how different" two arrays are.

Here you can find the Python implementation of the most common similarity measures.

$\endgroup$
0
$\begingroup$

Alex, what about trying clustering. It may give you more information about your dataset.

So you can start with sklearn.cluster.Kmeans that will take some users randomly as centers of clusters and then look for other similar users checking if the distance between their features is close to central users.

If you are sure that clusters may be of different size (in some clusters features are very close while in some clusters features can be more distributed) you can try sklearn.cluster.DBSCAN where you can as well apply different sklearn.metrics.pairwise_distances

$\endgroup$
0
$\begingroup$

I would use the unsupervised nearest neighbors (kNN) ; for instance in scikit-learn: https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.NearestNeighbors.html#sklearn.neighbors.NearestNeighbors

It allows you to choose between many distances (or similarity measures), including e.g. Jaccard similarity for categorical variables.

You can use the fit() method on your set, then return all K nearest neighbors to a given user with the kneighbors() method.

$\endgroup$
0
$\begingroup$

All you need to do is to develop a distance metric for every feature, that is a function which tells you how far apart two different elements of the same feature are. Once you have such a function for every feature, you can calculate the distance between users by calculating the sum between their features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.