# Tag Info

Accepted

### One Hot Encoding vs Word Embedding - When to choose one or another?

One-Hot Encoding is a general method that can vectorize any categorical features. It is simple and fast to create and update the vectorization, just add a new entry in the vector with a one for each ...
• 21.7k
Accepted

### Confusion about Entity Embeddings of Categorical Variables - Working Example!

For those who are interested, I've spent some time, finally figured out that the problem was the way one has to prepare the categorical encoding for the Entity Embedding suitable for a neural network ...
• 4,279

### What is the difference between and Embedding Layer and an Autoencoder?

Actually they are 3 different things (embedding layer, word2vec, autoencoder), though they can be used to solve similar problems. (i.e. dense representation of data) Autoencoder is a type of neural ...
• 850
Accepted

### Difference between Gensim word2vec and keras Embedding layer

Yep, you're right! As you know, it's difficult for machine learning models to use natural language directly, so it helps to transform words into some meaningful numeric representation. This process ...
• 2,734
Accepted

### Backpropgating error to emedding matrix

An embedding layer is in fact a linear layer. It maps the input, using a matrix multiplication, to the output, without any activation function after the multiplication. Therefore, the backpropagation ...
• 6,101

### One Hot Encoding vs Word Embedding - When to choose one or another?

It seems that Embedding vector is the best solution here. However, you may consider a variant of the one-hot encoding called 'one-hot hashing trick". In this variant, when the number of unique words ...
• 296

### word2vec word embeddings creates very distant vectors, closest cosine similarity is still very far, only 0.7

Let us try and understand how Word2Vector actually works before looking at distances: There are 2 ways of generating vectors for a word : Continuous bag of words Skip grams The following diagram ...
• 785

### Why are character level models considered less effective than word level models?

You are absolutely right about vocabulary size. I am actually conducting research on making character-level more effective. Here is why word-level tokens are often favoured despite, characters ...
• 6,076
Accepted

### Tensorflow: how to look up and average a different amount of embedding vectors per training instance, with multiple training instances per minibatch?

Use tf.gather(). Single instance case In the example below, we selected a variable number of embedding vectors from the matrix ...
• 2,284

### Auto-Encoder to condense (pre-process) large one-hot input vectors?

Leland's answer is exactly correct regarding why an autoencoder wouldn't be useful. Let me expand upon that point: Autoencoders and other dimensionality reduction techniques attempt to keep objects ...
• 2,381
Accepted

### Auto-Encoder to condense (pre-process) large one-hot input vectors?

It might be useful to think of this in terms of orthogonality. You state that "categories are not correlating in any way", which effectively means each category should be completely orthogonal to all ...
Accepted

### What is the neural network architecture behind Facebook's Starspace model?

Calling StarSpace a neural model would be misleading I think. You could certainly think of the it as a neural network with a single layer and a linear activation function, but I think don't think that ...
• 136
Accepted

### Choosing an embedding feature dimension

I think post below is a good resource. https://developers.googleblog.com/2017/11/introducing-tensorflow-feature-columns.html Basically, all categorical variable is initially converted to one-hot ...
• 191
Accepted

### Fastest way for 1 vs all lookup on embeddings

There are libraries that are specialized in exactly that task, for instance FAISS by Facebook AI Research: Faiss is a library for efficient similarity search and clustering of dense vectors. It ...
• 27k
Accepted

### Loss and Regularization inference

There is no problem with the fact that your regularization loss is going up. The cost function of your model is a weighted sum of the regularization loss and the base loss, so during training the ...
• 2,220

### Unordered Input

The suggestion of ncasas is a good one but not very clean. This ordering makes a lot of sense when it's 1-dimensional, but when you introduce more features the ordering will become more and more ...
• 9,378

### What is word embedding and character embedding ? Why words are represented in vector with huge size?

Most problems in NLP require the system to understand the semantic meaning of the text and not just the arrangement of specific words. Semantic understanding enables a system to say that, "I am ...
• 2,200

### What is word embedding and character embedding ? Why words are represented in vector with huge size?

There are several types of vector representations for words and characters, I'm assuming here the primary interest is dense representations that are used commonly in deep learning today. First, Some ...
• 31
Accepted

### Use embeddings to find similarity between documents

There are several ways you can obtain document embeddings. If you want to obtain a vector of a document that is not part of the trained doc2vec model, gensim provides a method called ...
• 349

### Flair Embeddings - Significance of Backwards vs Forwards?

Upon further research I found that during training forward language models try to predict the next word in a sequence. Backwards language models on the other hand start at the end of a sequence and ...

### What are the sparse and dense vector ? I cant undestand ,can you explain to me?please.Why do we use for?

A sparse matrix A sparse matrix is a matrix that is comprised of mostly zero values. e.g 1, 0, 0, 1, 0, 0 A = 0, 0, 2, 0, 0, 1 0, 0, 0, 2, 0, 0 A ...

### How much text is enough to train a good embedding model?

It is not the number of many articles that matter but the total number of words. Enough "meaningful/good" is an empirical question that depends on the dataset. One way to test the results of ...
• 21.7k

### What does the embedding mean in the FaceNet?

An embedding is a mapping from discrete objects, such as words, to vectors of real numbers. - Tensorflow/Embeddings With reference to the FaceNet paper, I should say that embedding here simply ...
• 2,425
Accepted

### What does the embedding mean in the FaceNet?

Assume you have features wich lie in a $R^n$ space, e.g. your input is a picture with $28 \times 28$ pixels, then $n$ would be $28 \times 28 = 784$. Now you can "embedd" your features into another \$...
• 931
Accepted

### What tasks you train with one set of features and predict with another?

If you mean completely new set of features for prediction: it will not be helpful. Your model 'learns' something on the training feature space and you hope to apply the learning to new data-points in ...
• 1,998

### Finding the best "depth" of ICD9 codes with pseudo-hierarchical clustering

It's called a prefix tree. And because the ICD codes are human made, and humans tend to think in categories, there is indeed such a structure in this codes. It's just that humans first thought of ...

### Retain similarity distances when using an autoencoder for dimensionality reduction

No it is not possible to preserve relative distances when reducing dimensions for arbitrary data. This is not due to a property of auto-encoders compared to e.g. PCA or T-SNE. It is due to geometry. ...
• 29k

### How to encode an array of categories to feed into sklearn

If the sequence of watches is meaningful, then you do need some kind of classifier that creates user/item embeddings from the sequence of watches, so you're probably looking at GRUs and LSTMs. Neural ...
• 6,595

### Are there any graph embedding algorithms like this already?

I would not be so skeptical. Yes, there is lots of research on random walk based recommender/ranking systems, including those similar to PageRank. Yes, spectral methods and linear algebra are ...
• 1,229