I am learning about matrix factorization for recommender systems and I am seeing the term latent features occurring too frequently but I am unable to understand what it means. I know what a feature is but I don't understand the idea of latent features. Could please explain it? Or at least point me to a paper/place where I can read about it?

  • $\begingroup$ Latent variables allow to render the models more powerful in terms what can be modeled. It's up to data and algorithm to define their value. In other words, latent variables are like "step" that bridges the gap between your observed variables and the desired prediction. The wider this "gap" is, the more useful the latent variables are. $\endgroup$ – Vladislavs Dovgalecs Oct 21 '15 at 0:28

At the expense of over-simplication, latent features are 'hidden' features to distinguish them from observed features. Latent features are computed from observed features using matrix factorization. An example would be text document analysis. 'words' extracted from the documents are features. If you factorize the data of words you can find 'topics', where 'topic' is a group of words with semantic relevance. Low-rank matrix factorization maps several rows (observed features) to a smaller set of rows (latent features). To elaborate, the document could have observed features (words) like [sail-boat, schooner, yatch, steamer, cruiser] which would 'factorize' to latent feature (topic) like 'ship' and 'boat'.

[sail-boat, schooner, yatch, steamer, cruiser, ...] -> [ship, boat]

The underlying idea is that latent features are semantically relevant 'aggregates' of observered features. When you have large-scale, high-dimensional, and noisy observered features, it makes sense to build your classifier on latent features.

This is a of course a simplified description to elucidate the concept. You can read the details on Latent Dirichlet Allocation (LDA) or probabilistic Latent Semantic Analysis (pLSA) models for an accurate description.

  • $\begingroup$ "Latent features are computed from observed features using matrix factorization." Is computation using matrix factorization a necessary condition for a quantity to be considered latent? $\endgroup$ – flow2k Jul 22 at 9:00

Suppose you have (MxN) sparse matrix, where M -- stands for number of users who gave recommendations, and N is the number of items recommended. The $x_{ij}$ element of the matrix is the recommendation given, with some elements missing, i.e. to be predicted.

Then your matrix can be "factorized", via introducing K "latent factors", so that instead of one matrix you have two: (MxK) --for users, and (KxN) -- for items, matrix multiplication of which produces the original matrix.

Finally, to your question: what are latent features in matrix factorization? They are unknown features (K) in user tastes and recommended items, so that when these two matrices multiplied, they produce matrix of known recommendations. Particular weights (of user preferences towards a particular feature and amount of a feature in a particular item) are defined via so called Alternating Least Squares algo, more about which you can read here


It seems to me that latent features is a term used to describe criteria for classifying entities by their structure, in other words, by features (traits) they contain, instead of classes they belong to. Meaning of the word "latent" here is most likely similar to its meaning in social sciences, where very popular term latent variable (http://en.wikipedia.org/wiki/Latent_variable) means unobservable variable (concept).

Section "Introduction" in the following paper provides a good explanation of latent features' meaning and use in modeling of social sciences phenomena: http://papers.nips.cc/paper/3846-nonparametric-latent-feature-models-for-link-prediction.pdf.

  • $\begingroup$ I read the introduction in the paper you referenced but didn't find it very useful in understanding the concept of latent features. $\endgroup$ – Will Oct 16 '17 at 10:17
  • $\begingroup$ @Will Feel free to suggest source(s) with better explanation. $\endgroup$ – Aleksandr Blekh Oct 16 '17 at 19:07
  • 1
    $\begingroup$ I quite like this one: tcts.fpms.ac.be/asr/project/sprach/report97/node162.html $\endgroup$ – Will Oct 17 '17 at 21:00
  • $\begingroup$ @Will Thank you. I agree - it is a pretty good introduction / explanation (though, I'm sure that there are many other good ones scattered out there). $\endgroup$ – Aleksandr Blekh Oct 18 '17 at 0:53

Another example, consider the case of users to movie rating matrix like the Netflix setup. This will be a huge sparse matrix which is difficult to process.

Note that each user will have a specific preference like sci-fi movies or romance movies etc. So, instead of storing all the movie ratings we could store a single latent feature like the movie category which belongs to different Genre's for example: sci-fi or romance, whichever quantifies his taste for each category. These are called Latent Features, which captures the essence of his taste rather than storing the entire movie list.

Of course this will be an approximation, but on the flip-side, you have a very little to store.

This is usually done using matrix decomposition techniques, like SVD which breaks an $N*N$ user to item recommendation matrix to $N*1$ user preference matrix and $1*N$ item preference matrix, added Advantage is that instead of storing $N^2$ number we effectively store $2N$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.