Questions tagged [openai-gpt]
The openai-gpt tag has no usage guidance.
11
questions
0
votes
0answers
18 views
Does GPT-2 has pre trained for sentiment analysis?
I tried sentiment analysis with 345M model of GPT-2. But it took a long time to train. So is there any other GPT-2 model available for sentiment analysis?
-1
votes
1answer
43 views
For NLP, is GPT-3 better than RoBERTa? [closed]
I am learning deep learning and I want to get into NLP. I have done LSTM, and now I am learning about vectorisation and transformers. Can you please tell me, which algorithm is more effective and ...
2
votes
1answer
23 views
How is GPT able to handle large vocabularies?
From what I understand, GPT and GPT-2 are trained to predict the $N^{th}$ word in a sentence given the previous $N-1$ words. When the vocabulary size is very large (100k+ words) how is it able to ...
1
vote
1answer
17 views
Generate text using user-supplied keywords
I've got a use case where I need to generate sentences based on a set of user supplied keywords. Here is an example of what I need:
User input:
End-User: Data Scientists
Region: Middle East
...
0
votes
0answers
11 views
How should we cut the long text into different sections for GPT-2 ? Do we use [PAD] to ensure the complete sentence included?
For GPT-2 model (I believe same for BERT) we need to cut the long text into fixed length for pre-training. Just wondering the details of how this should be implemented.
Do we need to cut the text ...
0
votes
2answers
142 views
Does the transformer decoder reuse previous tokens' intermediate states like GPT2?
I recently read Jay Alammar's blogpost about GPT-2 (http://jalammar.github.io/illustrated-gpt2/) which I found quite clear appart from one point :
He explains that the decoder of GPT-2 processes input ...
2
votes
2answers
33 views
How to generate a sentence with exactly N words?
Thanks to GPT2 pretrained model now it is possible to generate meaningful sequence of words with or without prefix. However a sentence should end with a proper endings (.,!,?). I am just wondering how ...
0
votes
0answers
26 views
Pretrained Models for Keyword-Based Text Generation
I'm looking for an implementation that allows me to generate text based on a pre-trained model (e.g. GPT-2).
An example would be gpt-2-keyword-generation (click here for demo). As the author notes, ...
1
vote
1answer
663 views
How does BERT and GPT-2 encoding deal with token such as <|startoftext|>, <s>
As I understand, GPT-2 and BERT are using Byte-Pair Encoding which is a subword encoding. Since lots of start/end token is used such as <|startoftext|> and , as I image the encoder should encode ...
3
votes
0answers
87 views
Fine tune gpt2 via huggingface API for domain specific LM
i am using the script in the examples folder to fine-tune the LM for a bot meant to deal with insurance related queries. So if someone were to type "i am looking to modify my ..." , the autocomplete ...
2
votes
0answers
140 views
Paragraph Generator using BERT or GPT
I am trying to generate similar sentences, called paragraph generation. For example, what is the name of the eldest brother of ram? - For these paragraphs can be - who is the oldest brother of ram? , ...
