3 votes
Accepted

Use embeddings to find similarity between documents

There are several ways you can obtain document embeddings. If you want to obtain a vector of a document that is not part of the trained doc2vec model, gensim provides a method called ...
Samarth's user avatar
  • 349
2 votes
Accepted

classification of similar text input features with text output label

I suggest you use the state of the art for this kind of problems: a BERT-based approach. This kind of approach is well documented and very accessible, given the large amount of examples available ...
noe's user avatar
  • 26.6k
2 votes

classification of similar text input features with text output label

That approach is reasonable. Short text inputs and multi-class classification outputs is a challenging problem. Genism's doc2vec hyperparameters probably matter less than collecting more data or ...
Brian Spiering's user avatar
2 votes

Approach to semantic similarity between documents

If I understand correctly, you're trying to map abstracts to their research papers. Here is a simple starting point: Compute a TF IDF model using the entire corpus (all abstracts + research papers). ...
Data's user avatar
  • 467
2 votes
Accepted

Difference between Doc2Vec and BERT

The main difference it that BERT includes attention mechanisms, whereas Doc2Vec doesn't. Attention mechanisms are functions to detect context between words, i.e. learning from words positions using ...
Nicolas Martin's user avatar
2 votes

How to examine if a Doc2Vec model is sufficiently trained?

One way to test an embedding space is to use word analogies as unit tests. A properly trained embedding space should successfully complete the analogy “Man is to king as woman is to _____" with &...
Brian Spiering's user avatar
1 vote

How to examine if a Doc2Vec model is sufficiently trained?

If you are unhappy with using your training-validation split set for evaluating your model, here are a few additional ways to compare your performance: Metric tracking. This is often used when data ...
Jacky Wong's user avatar
1 vote

Word2Vec vs. Doc2Vec Word Vectors

If you only care about word similarity, then apply Occam's Razor and use word2vec. There is no need to increase model complexity if not going to be used. Also, the quality of embeddings is primarily ...
Brian Spiering's user avatar
1 vote

Clustering using both text and numerical features

One way to achieve this is to use a clustering method based on a custom similarity/distance measure. For example you could defined the similarity measure between two instances as: $$sim(\langle v_1, ...
Erwan's user avatar
  • 25.3k
1 vote

DBSCAN on textual and numerical columns

You did not mention which package you are using. If you using scikit-learn, sklearn.pipeline.FeatureUnion concatenates results of multiple transformer objects. ...
Brian Spiering's user avatar
1 vote

Word2Vec with CNN

The idea is that Doc2Vec is an average of all your word vectors. Your document embedding will be the same size as any other word vector you have. If you average 10 word embeddings of size ...
Bruno Lubascher's user avatar
1 vote

Word2Vec with CNN

I think RNNs (LSTM, GRU etc.) is more suitable for text classification. CNNs has a fixed window size, thus you will always need to pad the input into same length. I took a cursory look at the ...
ColinDowney's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible