1
$\begingroup$

Suppose I'm making a movie recommendation engine and I have tons of data about the movies like genre, ratings, release info, earnings, etc. It makes sense to create an embedding layer for the users since I have no additional info on them and the embeddings will learn the geometry of the users based on their watch preferences. My gut still tells me to make an embedding layer for the movie index as well and concatenate it with the movie-related numerical features. Why though?

Can't we just pass in the numerical features of the movies for the model to differentiate? Are the embeddings supposed to capture some additional latent information perhaps not captured in the raw metadata?

I know the embeddings get us the nice feature of being able to compare similar movies but I could just do the same this with my numerical features right?

This isn't specific to recommendations but generally, why would you create an embedding vector for categorical features when you already have a lot of descriptive data about each item in that categorical feature?

$\endgroup$

2 Answers 2

0
$\begingroup$

It is important what the numbers in a model mean. Embeddings capture the semantic relationships between entities as distance. This useful for machine learning because algorithms can learn how predict a label based on distance, thus learn how semantic relationships between entities are predictive.

$\endgroup$
0
$\begingroup$

We create embeddings to have the dense representation of the sparse feature vectors. Embeddings map the high dimensional data into a low dimensional space.

Note

A dense feature is the one which has mostly non zeros in it whereas a sparse feature is the one which has mostly zeros in it.

Q: My gut still tells me to make an embedding layer for the movie index as well and concatenate it with the movie-related numerical features. Why though?

Yes, we can do that. Lets say for a particular user "U" we want to pass all the movies he has watched as a feature. We have a total of 10,000 movies in our dataset and the user "U" watched only 10 movies. One way we can achieve this is the "one hot encoding" that is put "1" at the index of the movie which the user "U" watched and put "0" everywhere else. This will create a 10,000 dimensional sparse feature with mostly zero values. This is a significant computation and memory overhead. This is where the "Embeddings" come into play.

"Judge a movie by the company it keeps". As Brian's answers says that the embeddings capture the sematic relationships between entities(movies or users) in our example. There are algorithms which generates embeddings for the input we pass word2vec, glove etc.

If you are curious there is something call "Embedding layer" that is we generate the embeddings with the neural network architecture.

Q: Can't we just pass in the numerical features of the movies for the model to differentiate? Are the embeddings supposed to capture some additional latent information perhaps not captured in the raw metadata?

As per my understanding the embeddings are only useful for categorical features and yes embeddings capture some more information as compare to say a normal categorical feature. Lets think in terms of higher dimensions. When we have simple feature (e.g genre) we need a one hot encoded feature vector to encode the genre of the movie but when we convert this into lets say 2 dimensional dense representation then we can point this feature on a 2D plane. Each point in the 2D plane (x, y) will represent a vector for the movie. Please note that by looking at the point on the 2D we cannot exactly say that what is the genre for that movie it will be just a vector in 2D plane i.e two numbers(x,y). But now we can find the similarity between two movies by computing the distance between their vector representations. This helps the machine learning/ DL model to capture the semantic relationships.

I hope that it was helpful. Thanks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.