As of May 31, 2023, we have updated our Code of Conduct.
3 votes
Accepted

Easy question on autoregressive LLM

At t=1: K, Q, V are the projections of the sequence "today is a good". However, given that the computations of the first tokens were already done in the previous step, usually, there is some ...
noe's user avatar
  • 20.5k
2 votes

Transformers doubt

We should not mistake the K, Q and V vectors received by the multi-head attention block with those received by the scaled dot-product block. The K, Q and V vectors that are fed to the multi-head ...
noe's user avatar
  • 20.5k
1 vote

Sentiment Analysis on the first 100 words of a very large essay of 500/700 words

Are there any potential issues on performing sentiment analysis using the first 100 words? No. You could do it and probably get away with it in certain instances. But if proper/accurate results are ...
spectre's user avatar
  • 1,576
1 vote

Load an LLM in multiple GPUs

Using multiple GPUs usually means that the whole model is copied into the memory of each of them. In Pytorch this is achieved with nn.DataParallel or nn.parallel.DistributedDataParallel. This however, ...
noe's user avatar
  • 20.5k
1 vote
Accepted

computer vision transformers: ViT does not have a decoder?

It depends on the task you want to perform. The goal here is to find a way to represent your image as a sequence of embedding vectors representing each patch of the image Once you have obtained an ...
Ciodar's user avatar
  • 142
1 vote
Accepted

How is the classification step performed in Swin Transformer?

There are many ways of doing this, and the authors did not specify it on purpose. One can aggregate the output vectors in various ways, or just choose the first position (like in BERT) solely to be ...
Johncowk's user avatar
  • 195
1 vote

detect/generate possible tokens for the dataset (name,type/category,signatures)

Given the nature of the problem, it might not be amenable to machine learning. The structure of the data drives how it can be modeled. The features are "name" (assumed to be a string) and &...
Brian Spiering's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible