# Meaning of latent features?

I am learning about matrix factorization for recommender systems and I am seeing the term latent features occurring too frequently but I am unable to understand what it means. I know what a feature is but I don't understand the idea of latent features. Could please explain it? Or at least point me to a paper/place where I can read about it?

• Latent variables allow to render the models more powerful in terms what can be modeled. It's up to data and algorithm to define their value. In other words, latent variables are like "step" that bridges the gap between your observed variables and the desired prediction. The wider this "gap" is, the more useful the latent variables are. Oct 21 '15 at 0:28

At the expense of over-simplication, latent features are 'hidden' features to distinguish them from observed features. Latent features are computed from observed features using matrix factorization. An example would be text document analysis. 'words' extracted from the documents are features. If you factorize the data of words you can find 'topics', where 'topic' is a group of words with semantic relevance. Low-rank matrix factorization maps several rows (observed features) to a smaller set of rows (latent features). To elaborate, the document could have observed features (words) like [sail-boat, schooner, yatch, steamer, cruiser] which would 'factorize' to latent feature (topic) like 'ship' and 'boat'.

[sail-boat, schooner, yatch, steamer, cruiser, ...] -> [ship, boat]

The underlying idea is that latent features are semantically relevant 'aggregates' of observered features. When you have large-scale, high-dimensional, and noisy observered features, it makes sense to build your classifier on latent features.

This is a of course a simplified description to elucidate the concept. You can read the details on Latent Dirichlet Allocation (LDA) or probabilistic Latent Semantic Analysis (pLSA) models for an accurate description.

• "Latent features are computed from observed features using matrix factorization." Is computation using matrix factorization a necessary condition for a quantity to be considered latent? Jul 22 '19 at 9:00
• The concept appears elsewhere, like Boltzman machines Aug 25 at 4:13

Suppose you have (MxN) sparse matrix, where M -- stands for number of users who gave recommendations, and N is the number of items recommended. The $x_{ij}$ element of the matrix is the recommendation given, with some elements missing, i.e. to be predicted.

Then your matrix can be "factorized", via introducing K "latent factors", so that instead of one matrix you have two: (MxK) --for users, and (KxN) -- for items, matrix multiplication of which produces the original matrix.

Finally, to your question: what are latent features in matrix factorization? They are unknown features (K) in user tastes and recommended items, so that when these two matrices multiplied, they produce matrix of known recommendations. Particular weights (of user preferences towards a particular feature and amount of a feature in a particular item) are defined via so called Alternating Least Squares algo, more about which you can read here

• the link is broken Aug 25 at 4:13

Another example, consider the case of users to movie rating matrix like the Netflix setup. This will be a huge sparse matrix which is difficult to process.

Note that each user will have a specific preference like sci-fi movies or romance movies etc. So, instead of storing all the movie ratings we could store a single latent feature like the movie category which belongs to different Genre's for example: sci-fi or romance, whichever quantifies his taste for each category. These are called Latent Features, which captures the essence of his taste rather than storing the entire movie list.

Of course this will be an approximation, but on the flip-side, you have a very little to store.

This is usually done using matrix decomposition techniques, like SVD which breaks an $N*N$ user to item recommendation matrix to $N*1$ user preference matrix and $1*N$ item preference matrix, added Advantage is that instead of storing $N^2$ number we effectively store $2N$.

It seems to me that latent features is a term used to describe criteria for classifying entities by their structure, in other words, by features (traits) they contain, instead of classes they belong to. Meaning of the word "latent" here is most likely similar to its meaning in social sciences, where very popular term latent variable means unobservable variable (concept).

Section "Introduction" in this paper provides a good explanation of latent features' meaning and use in modeling of social sciences phenomena.

• I read the introduction in the paper you referenced but didn't find it very useful in understanding the concept of latent features.
– Will
Oct 16 '17 at 10:17
• @Will Feel free to suggest source(s) with better explanation. Oct 16 '17 at 19:07
• I quite like this one: tcts.fpms.ac.be/asr/project/sprach/report97/node162.html
– Will
Oct 17 '17 at 21:00
• @Will Thank you. I agree - it is a pretty good introduction / explanation (though, I'm sure that there are many other good ones scattered out there). Oct 18 '17 at 0:53