0
$\begingroup$

I am working on a sentiment analysis project. I used BERT model for training but lack of data it gives huge overfitting. So after i moved LLM approach to do that. Using LLM finally i got good results. But sometimes it gives wrong outputs. So now i want to use some Reinforcement leaning approach. If model gives correct answer, user give thumbs up or else thumbs down like that. I want some guide to do that. I'm using gpt 3.5 turbo as my model.

$\endgroup$
2
  • $\begingroup$ you can use RLHF, but sentiment analysis is usually a classification problem, so you can just interpret the feedback as target, and train the model as usual $\endgroup$ Commented Nov 6, 2023 at 13:50
  • $\begingroup$ Can i use RLHF with OpenAi models. Like gpt-3.5. $\endgroup$ Commented Nov 8, 2023 at 14:30

1 Answer 1

1
$\begingroup$

This could be achieve by fine-tuning the model using labeled data (prompts + expected answers). Open AI API makes it possible to fine-tuning chat GPT models as explained here but it is quite expensive as you would be using some hardcore hardware to do this and has to be done very carefully in order to achieve good results.

I would suggest you to analyze which cases chat gpt is giving you bad results and developing some initial prompt that gives the model enough context so it doesn't make the same mistake.

Example: If you are using chat gpt in a soccer specific context and it is saying that a soccer match is composed of 20 players, you could start every prompt saying something like: "You are a chat bot developed to answer questions about soccer precisely. Don't forget that a soccer team has 11 players in total, counting the goalkeeper."

Obviously this example is a bit extreme but it gives the idea. Otherwise, if mistakes are a lot more complex than that and fine tuning is really necessary, I would suggest you do some research on LLMs fine tuning and evaluate open source models possibilities in order to avoid cost.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.