Easy question on autoregressive LLM
At t=1: K, Q, V are the projections of the sequence "today is a good". However, given that the computations of the first tokens were already done in the previous step, usually, there is some ...
We should not mistake the K, Q and V vectors received by the multi-head attention block with those received by the scaled dot-product block. The K, Q and V vectors that are fed to the multi-head ...
Sentiment Analysis on the first 100 words of a very large essay of 500/700 words
Are there any potential issues on performing sentiment analysis using the first 100 words? No. You could do it and probably get away with it in certain instances. But if proper/accurate results are ...
Load an LLM in multiple GPUs
Using multiple GPUs usually means that the whole model is copied into the memory of each of them. In Pytorch this is achieved with nn.DataParallel or nn.parallel.DistributedDataParallel. This however, ...
computer vision transformers: ViT does not have a decoder?
It depends on the task you want to perform. The goal here is to find a way to represent your image as a sequence of embedding vectors representing each patch of the image Once you have obtained an ...
How is the classification step performed in Swin Transformer?
There are many ways of doing this, and the authors did not specify it on purpose. One can aggregate the output vectors in various ways, or just choose the first position (like in BERT) solely to be ...
detect/generate possible tokens for the dataset (name,type/category,signatures)
Given the nature of the problem, it might not be amenable to machine learning. The structure of the data drives how it can be modeled. The features are "name" (assumed to be a string) and &...
Only top scored, non community-wiki answers of a minimum length are eligible