Skip to main content
6 votes
Accepted

Why both ChatGPT and Bard can't get a simple matrix calculation right?

In my understanding, LLMs are, very simplified speaking, probabilistic solvers. Math problems such as matrix multiplications are, on the other hand, deterministic in nature. Thus, using a an LLM for ...
Eulenfuchswiesel's user avatar
4 votes
Accepted

Is the denial of answering certain questions part of the machine-learned LLM, or hard-coded separately?

The efforts to add "guardrails" to LLMs is usually referred to as LLM alignment. While the internals of ChatGPT are not known, usually LLM alignment is done at the model level via training/...
noe's user avatar
  • 27k
4 votes
Accepted

what is the difference between window size and context length of language model?

Yes, in large language models, window and context length refer to the same thing: the maximum token sequence length that the language model can handle at once.
noe's user avatar
  • 27k
3 votes
Accepted

Why do GPT models use a transpose of the embedding matrix to convert outputs to logits?

As you described in your comment, this technique regularizes the model by reusing the same weights for the logit computation. I think, however, that there is previous reference to this technique prior ...
noe's user avatar
  • 27k
2 votes
Accepted

How to measure accuracy of GPT model

You can check the Question Generation section of paperswithcode. There, you can see for different datasets how the performance is measured and how different proposed approaches compare on them. ...
noe's user avatar
  • 27k
2 votes

Do we really need a very large dataset to train GPTs?

Train from scratch If you want to train a LLM from scratch, then its a different ball game. In this case, yes you need a lot of data for training purposes. This is because when you fine tune a model (...
spectre's user avatar
  • 2,115
1 vote

Searching ChatGPT transcripts?

Had asked on Reddit too. Didn't expect an answer, but Easy ChatGPT folders was suggested. It has full chat history search and nested folders. Also bulk delete. Rating of 4.5 from 21 reviews, and 2000 ...
PracticalKat's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible