Skip to main content

Questions tagged [gpt]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

does DALL-E api use Microsoft account info while generating responses?

I have read some work regarding occupational gender bias in AI image generation and it seemed. According to my research, tools like DALL-E generated more images of men when trying out for images with ...
x89's user avatar
  • 191
1 vote
1 answer
95 views

How OpenAI embeddings work?

I was looking at the Stanford CS224N NLP with Deep Learning lecture, and in the first two videos, we are introduced to word2vec models. The high-level idea mentioned was that we have a 'big corpus' of ...
mw981's user avatar
  • 13
0 votes
0 answers
13 views

Tips for pre-processing long unstructured text?

I'm working on a React-Node project where I'm trying to pre-process some text before passing it to GPT-4 to perform information extraction out of it, the flow of the project is: User uploads document ...
Abdul Munem's user avatar
4 votes
1 answer
1k views

How do I prompt GPT-4 to look at a PDF in Jupyter Notebook?

I am a beginner. I purchased tokens to use GPT-4 and finally figured out how to import the GPT-4 model into my Jupyter Notebook. ...
Mas's user avatar
  • 55
1 vote
1 answer
324 views

How is openAI embedding models trained?

how it the embedding model trained? Are the embeddings simply extracted from chatGPT4 or are they trained differently from the beginning (pre-training stage)?
haneulkim's user avatar
  • 469
0 votes
0 answers
56 views

Stream response from custom RASA actions to the chatbot

I am using RASA PRO with CALM. I was thinking of using openai api within a custom action and stream the streaming response coming from openai to my chatbot. Openai is giving me streaming response and ...
Avatar's user avatar
  • 1
0 votes
0 answers
14 views

What is system prompt in fine tuning of GPT3.5 for natural language to sql queries

What exactly is a system prompt while finetuning GPT3.5 or a language model in general? How can I build system prompt for the task of converting natural language to SQL queries
heyula's user avatar
  • 37
0 votes
1 answer
34 views

What ML model is best suited for an intelligent search assistant?

I'm working on my thesis project, and want to make an intelligent search assistant that understands context and, of course, processes and repsonds in natural languaje. The data I want to train this ...
Bito's user avatar
  • 1
0 votes
1 answer
19 views

predict next career suggestion

I have a dataset having job and description. i want to make model which can predict what are the thing that user needs to improve when the user inputs his skills. For an example, If he has skills - ...
pycoder's user avatar
1 vote
0 answers
60 views

How can I use Time-GPT for pretraining my model

I am mentioning Time-GPT here as a placeholder example. It can be any pretrained model. Suppose I have a dataset that requires some time series prediction. How can I leverage a well-trained model and ...
Mohammad Mosiur's user avatar
0 votes
0 answers
12 views

Create a gpt-3.5 API request that determines whether any time range in a list intersects with a given time range

I've created a prompt that should select a requested number of employees from the list. But the step 1 doesn't work properly. Sometimes GPT takes in account only the time range and ignores the date. I ...
pazukdev's user avatar
  • 101
0 votes
0 answers
18 views

How are GPT2 token embedding vectors processed internally?

I am experimenting with the GPT2-XL model and trying to understand the internal structure. While I understand most of the components and how they affect the size of the activation tensors (such as ...
John Doe's user avatar
1 vote
1 answer
44 views

Falcon-7B llm giving random output

I am using a falcon 7B model for a chatbot without any finetuning with the following code ...
Saket Vempaty's user avatar
0 votes
1 answer
81 views

Do we really need a very large dataset to train GPTs?

Do we really need a very large dataset to train GPTs? If this dataset is not big, won't GPT work well? Or will it still work better than conventional learning models in this situation? And is it ...
nisar's user avatar
  • 1
0 votes
0 answers
29 views

data verctorizing of a free text field to get better responses of the prompt

wonder if best practices are existing to approach the solution "what kind of vectorizing of a input filed" is the best for my use-case. Model, size and others can have a significant impact ...
eric's user avatar
  • 1
0 votes
1 answer
44 views

Reinforcement learning

I am working on a sentiment analysis project. I used BERT model for training but lack of data it gives huge overfitting. So after i moved LLM approach to do that. Using LLM finally i got good results....
Sandun Tharaka's user avatar
0 votes
1 answer
224 views

What is the "Extract" token and how is the final Linear layer applied in GPT?

In the manuscript of GPT, the authors have given the following image: Questions: What is the final "Extract" (token?)? Is it the "END" token? How is the final linear layer ...
figs_and_nuts's user avatar
1 vote
0 answers
45 views

Is openAI text generation models an extension of embedding models?

we can creating embeddings using below code ...
Vinay Sharma's user avatar
1 vote
1 answer
272 views

A question about contextual embeddings in the decoder only transformer architecture (gpt)

I am reading up on the decoder only architecture Relevant excerpts: We can use any model that maps token sequences into contextual embeddings (e.g., LSTMs, Transformers): $$\phi : V^L \to R^{d \times ...
figs_and_nuts's user avatar
2 votes
1 answer
648 views

what is the difference between window size and context length of language model?

is window size and context length of language model one and the same thing? ******** following text is added as question with ONLY above text was not allowed ***** I am trying to understand how GPT ...
Vinay Sharma's user avatar
1 vote
0 answers
195 views

How does supervised fine-tuning work in InstructGPT?

See Figure 2 from the InstructGPT paper: I want to know how Step 1 works. Here is one possible algorithm. Pass the prompt through the model, and compute the negative log of the probability of the ...
jskattt797's user avatar
0 votes
1 answer
541 views

Why do GPT models use a transpose of the embedding matrix to convert outputs to logits?

According to section 3.1 of the original GPT paper, GPT right-multiplies the final output vectors (after applying a Transformer decoder model) by the transpose of the embedding matrix, before applying ...
jskattt797's user avatar
7 votes
3 answers
10k views

Why does everyone use BERT in research instead of LLAMA or GPT or PaLM, etc?

It could be that I'm misunderstanding the problems space and the iterations of LLAMA, GPT, and PaLM are all based on BERT like many language models are, but every time I see a new paper in improving ...
Ethan's user avatar
  • 173
1 vote
1 answer
638 views

How to measure accuracy of GPT model

I am working on a model to build questions automatically from some text My model will analyse provided article and ask authors questions that can help improving their articles How can we measure the ...
asmgx's user avatar
  • 549
0 votes
0 answers
167 views

Disadvantages of using just Vector Stores

I've just started with langchain, and one thing baffles me. For starters, I'm using an in-memory vector store created from a local csv file ...
dejanualex's user avatar
0 votes
0 answers
60 views

What is the input and output of GPT model for fine-tuning?

From my understanding, for the pretraining of GPT model, we need to do next token prediction task. In this case, Input -> The GPT models are general-purpose language models that can perform ... (...
Kyuwan's user avatar
  • 1
2 votes
1 answer
146 views

How was the token library constructed for ChatGPT / other GPT systems?

I have found literally hundreds of articles on Google with titles like 'What are tokens and how to use them,' but haven't been able to find any information at all on how the token libraries themselves ...
Sinnombre's user avatar
  • 153
0 votes
0 answers
30 views

Are there any studies on the actual loss of quality in a quantized GPT model? (eg GPTQ)

I've been trying to think of how different a GPT would perform if we swapped from floats to 8-bit. Given that we take say the top 10 words, how much fidelity is actually lost? How many tokens are ...
JuanIsFree's user avatar
2 votes
2 answers
590 views

Training GPT Model with Swagger Documents, Need Help with Model Fit

We are currently developing an application that performs actions based on user input using 'gpt-3.5-turbo-0613'. For example, if a user enters the prompt "create a user with the name Potter and ...
mostwelcome's user avatar
1 vote
1 answer
98 views

Why is the target output in GPT (Andrej Karpathy's course) contains copies of tokens from the input?

The question is based on Andrej Karpathy lecture on training a toy GPT model on Tiny Shakespeare dataset (youtube link). In this model, tokens are single characters with a dictionary of around 60 ...
DeLorean88's user avatar
1 vote
0 answers
742 views

Creating LLM chatbot using llama-index + langchain

As the title suggests: I'm trying to build a chatbot which his goal should be sort of like "chatgpt". The chatbot will be installed on Slack workspace and I'm struggling with which scope I ...
Ben's user avatar
  • 209
0 votes
1 answer
258 views

Passing target text to gpt2 and T5 for fine tuning to learn text generation task

I have text with each line in following format: <text-1> some text-1 <text-2> some text-2 <text-3> some text-3 I want fine tune model to learn ...
RajS's user avatar
  • 103
2 votes
1 answer
478 views

dolly v2 - how does it internally learn to follow instructions

this is more a curiosity query than anything else. The git repo for dolly gives us an easy way to swap, the training dataset , to train custom models, as long as we follow the format. I however have ...
Vikram Murthy's user avatar
0 votes
1 answer
20 views

Where can I see a summary of tools and techniques for most updated transformer developments?

Since the invention of chatGPT, there are many tools and techniques and variants invented ever since. I want to keep track of these developments and tools but I find no avail. May I know which ...
Student's user avatar
  • 411
1 vote
1 answer
4k views

Chromadb in my local system

I am trying to install chromadb on my local system but getting following error even after installing Microsoft C++ Build Tools: Is it because "Chromadb" can only be used in google Collab and ...
Spandan Bhatnagar's user avatar
3 votes
1 answer
247 views

Use a "ChatGPT like" engine really locally without exposing sensitive data

I was recently playing around with llama_index and llama Hub and found it very easy to be used on my own data set. While this is ...
Ben's user avatar
  • 209
15 votes
1 answer
15k views

How does an LLM "parameter" relate to a "weight" in a neural network?

I keep reading about how the latest and greatest LLMs have billions of parameters. As someone who is more familiar with standard neural nets but is trying to better understand LLMs, I'm curious if a ...
slim_wizard's user avatar
1 vote
0 answers
21 views

In ChatGPT, The difference of using reward to guide policy vs using the dataset of reward to train policy?

In ChatGPT, What are the differences of using reward to guide policy vs using the dataset of reward to train policy?
CoderOnly's user avatar
  • 711
5 votes
5 answers
1k views

Is using GPT-4 to label data advisable?

If I have a lot of text data that needs to be labeled (e.g. sentiment analysis), and given the high accuracy of GPT-4, could I use it to label data? Or would that introduce bias or some other issues?
cookiecutter's user avatar
1 vote
1 answer
126 views

What should the numerical values of the <startofsentence> and <endofsentence> token vectors be?

I'm trying to build GPT2 from scratch. I understand how to convert each word in a sentence to its respective token index and each token is then converted to its respective word embedding vector. I ...
Austin Capobianco's user avatar
0 votes
1 answer
427 views

What are the benefits of changing the temperature parameter in the chatgpt api?

(Cross-posted from: https://ai.stackexchange.com/q/39813/57463 after a week of no satisfactory answer) TLDR: What are the benefits of increasing the temperature in the chatGPT API ? Context (can be ...
userrandrand's user avatar
2 votes
1 answer
2k views

Can I use LLM to explain codebase?

I am a Data Engineer, and I am currently assigned a task to refactor an outdated code and rectify any bugs present. However, I am unable to comprehend the code written in the existing codebase. ...
Yiffany's user avatar
  • 158
3 votes
1 answer
1k views

How do GPT models go from token probabilities to textual outputs?

Suppose GPT-2 or GPT-3 is trying to generate the next token, and it has a probability distribution (after applying softmax to some output logits) for the different possible next tokens. How does it ...
jskattt797's user avatar
0 votes
1 answer
105 views

Where can I find gpt-35-turbo in the Microsoft Pricing calculator?

Where can I find gpt-35-turbo in the Microsoft Pricing calculator? I don't see gpt-35-turbo in the model list: https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/ ...
Franck Dernoncourt's user avatar
4 votes
2 answers
17k views

ChatGPT: How to use long texts in prompt?

I like the website chatpdf.com a lot. You can upload a PDF file and then discuss the textual content of the file with the file "itself". It uses ChatGPT. I would like to program something ...
meyer_mit_ai's user avatar
0 votes
1 answer
348 views

GPT-2 architecture question

I am currently working on a NLP model that compares two comments and determines which one would be more popular. I have already came up with an architecture - it will be based on GPT-2. But now I am ...
Euler-Maskerony's user avatar
0 votes
1 answer
1k views

What are the advantages of autoregressive over seq2seq?

Why are recent dialog agents, such as ChatGPT, BlenderBot3, and Sparrow, based on the decoder architecture instead of the encoder-decoder architecture? I know the difference between the attention of ...
Fei Wang's user avatar
0 votes
1 answer
566 views

Words limit using GPT-3 API and fine tune model

In the documentation for GPT-3 API, it says One limitation to keep in mind is that, for most models, a single API request can only process up to 2,048 tokens (roughly 1,500 words) between your prompt ...
Salty Gold Fish's user avatar
2 votes
1 answer
236 views

Does GPT-3 remember data from prompts used to fine tune it?

I am trying to fine tune a model using OpenAI's fine tuning API. I am passing bodies of text (for example, news paper articles) as prompts and the data I want from it as completions. Let us consider ...
Kushal Mohnot's user avatar
3 votes
1 answer
2k views

Does OpenAI and ChatGPT use Scikit Learn?

Scikit Learn is Python's go-to open source package for running common AI and machine learning algorithms. Does OpenAI and its product ChatGPT use or rely on Scikit Learn on its back-end at all? If not,...
user610620's user avatar