1
$\begingroup$

I have users in my database that I would like to match up or group togetter based on the content of there articles. I cant seem to find how this kind of problem is being solved today. Any advice will help.

Available Data:

1) Each user's posts (anything written by them, like a blog).

2) Tags for each post (tags that the user gave to their post when they created it)

Goal:

1) Match/Group users based on available data.

2) Produce match percentage.

Attempt:

I matched people based on the number of exact matches of their tags.

Example: user1 has [car,honda,sports], user2 has [car,food].

This will give a 33% match.

As you can image this does not work very well. Most users have 20 tags but typically get a match percentage of 0% even if they are talking about similar things.

Problem:

Tags that have a clear relationship like CAR and HONDA are NOT matched.

Question:

How can I match/Group users based on tags or the content of there articles?

$\endgroup$
1
$\begingroup$

One way could be to apply word-embeddings for semantic similarity checking. word2vec model generates feature vectors which could capture semantic similarity. For example, the closest vector to car will be honda, ferrari, vehicle, bike. Train a model using large amount of data from wikipedia dumps or the one released by google. It has fine quality vectors. Gensim has a nice implementation of word2vec

For each blog articles, pre-process the data by removing the stop-words and stemming them. From the resulting words, collect the more frequent words. Do this for all the articles and check for the similarity among the frequent words in other articles as well. So that one article with frequent words car, race, tournament, ferrari, F1 will have more closer vectors in article with frequent words bike, honda, racer.

Or other way is to look for similar vectors in tags itself. It is good to play with it for sometime, so that you get to know which features work better for the dataset you have.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.