I am trying to find more information about pre-trained model distilbert-base-nli-mean-tokens
. Can someone please point me to it's paper or documentation? Is it based on DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter paper? This is published in March 2020. I am looking for links between this paper and Sentence-BERT (sentence-transformers). Original sentence-bert paper is published in Aug 2019. I wanted to try pre-trained model using S-BERT model and hence tried distilbert-base-nli-mean-tokens
model. After implementation I found out that it's much faster than other pre-trained models available on sentence-transformer website. While studying it's paper I realised original paper do not mention this pre-trained model.
I found Making Monolingual Sentence Embeddings Multilingual using
Knowledge Distillation this paper published by same author which do mention DistilmBERT
but not DistilBert
Can someone please help me solve this mystery?