I want to create a document ranking model which returns similar rows in the dataset for a sample query. The text in this corpus is standard english but without any labels (ie no query-related documents structure). Is it possible to use a pretrained model trained on a large corpus (like bert or word2vec) and use it directly on the scraped dataset without any evaluation and get decent results? If not this, is training a model on the MS macro dataset and applying it on this corpus worth exploring?
3
-
1$\begingroup$ can you please give us more details about the ranking criteria? $\endgroup$– MXKCommented Jul 6, 2021 at 16:25
-
1$\begingroup$ If I understand correctly it looks like information retrieval, if so the idea would be to calculate a similarity score between each row/document and the query. No need for labels. $\endgroup$– ErwanCommented Jul 6, 2021 at 22:41
-
1$\begingroup$ Yes, i want to calculate similarity score between the rows of the dataset and the query. Thanks for the answer! I was under the impression that we might need to feed labels as well to the model for this task. $\endgroup$– sarvaCommented Jul 7, 2021 at 4:27
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
It depends on the type of ranking that you want to achieve, for example if the unlabeled scraped data can be ranked by sentiment, you can use Transfer Learning models to give each document a sentiment score which will serve as a rank if you return the sentiment score probability instead of having "positive" and "negative" tags.
Transfer Learning models usually give a good result but it's really up to your criteria for ranking the documents, and you should pay attention to the quality of the scraped data, it affects heavily the pre-trained model results.
Now since you have mentioned MS macro dataset, i'm assuming that your documents are maybe related to Question and Answer datasets, I think you should also take a look at The Stanford Question Answering Dataset.
-
1$\begingroup$ I want to rank the rows based on how (using either word embeddings or contextual embeddings) similar it is text-wise (or convey similar meaning) to the query. Would we be needing labels for that? $\endgroup$– sarvaCommented Jul 7, 2021 at 4:39
-
$\begingroup$ In this case I think you can use Use the tfidf. rank_document() function to score documents based on overlapping content or Use the docsim. DocSim() class to score documents on similarity using doc2vec and the GloVe word embedding model. $\endgroup$– MXKCommented Jul 7, 2021 at 9:14