Questions tagged [language-model]
Language models are used extensively in Natural Language Processing (NLP) and are probability distributions over a sequence of words or terms.
8 questions
49
votes
7
answers
58k
views
What is the difference between model hyperparameters and model parameters?
I have noticed that such terms as model hyperparameter and model parameter have been used interchangeably on the web without prior clarification. I think this is incorrect and needs explanation. ...
- 2,157
17
votes
7
answers
23k
views
ChatGPT's Architecture - Decoder Only? Or Encoder-Decoder?
Does ChatGPT use an encoder-decoder architecture, or a decoder-only architecture? I have been coming across Medium and TowardsDataScience articles suggesting that it has an encoder-decoder ...
- 321
73
votes
4
answers
109k
views
What is purpose of the [CLS] token and why is its encoding output important?
I am reading this article on how to use BERT by Jay Alammar and I understand things up until:
For sentence classification, we’re only only interested in BERT’s output for the [CLS] token, so we ...
- 947
18
votes
2
answers
34k
views
Word2Vec embeddings with TF-IDF
When you train the word2vec model (using for instance, gensim) you supply a list of words/sentences. But there does not seem to be a way to specify weights for the words calculated for instance using ...
- 301
16
votes
6
answers
8k
views
Are there any good out-of-the-box language models for python?
I'm prototyping an application and I need a language model to compute perplexity on some generated sentences.
Is there any trained language model in python I can readily use? Something simple like
<...
- 403
10
votes
5
answers
19k
views
How to create a good list of stopwords
I am looking for some hints on how to curate a list of stopwords. Does someone know / can someone recommend a good method to extract stopword lists from the dataset itself for preprocessing and ...
- 333
3
votes
1
answer
2k
views
Fine-tuning a pre-trained LLM for question-answering
Objective
My goal is to fine-tune a pre-trained LLM on a dataset about Manchester United's (MU's) 2021/22 season (they had a poor season). I want to be able to prompt the fine-tuned model with ...
- 33
1
vote
1
answer
171
views
Some answers given by ChatGPT are just beyond ridiculous, what could be the reasons?
Some answers given by ChatGPT are just beyond ridiculous, especially in Chinese (I am Chinese so I ask ChatGPT in both Chinese and English). Here is an example,
The last answer
...
- 133